sipXconfig

Things people are eager to tell you...

2007-12-06T12:56:00.001-08:00

Damian: I noticed that if one writes on the list "I am going to do this and this"
Damian: it's better than "what should I do?"
Kevin: ok
Kevin: people are always eager to tell someone they are a moron, but not as eager to answer questions :)

Sketches of Frank Ghery

2007-11-07T07:51:00.000-08:00

Coding is not like building houses. The two are often compared. However, this comparison usually results in all kinds of design recommendation that effectively prohibit producing any usable software.

If you want to see that metaphor reversed make sure to watch Sketches of Frank Ghery movie. It shows the architect who design buildings the way that software should be written. There are of course beautiful buildings to watch: Disney Concert hall, Bilbao Guggenheim.

It's a movie about architecture, but more than that it's a movie about design process: iterative modeling, refactoring, debugging - it is all there. Gehry is definitely into refactoring. And into small iterations. It's fun to watch when Gehry and his team apply programming terminology to the models they create. Sidney Pollack could not possibly make a movie about a coders, but only because bunch of geeks arguing on the list or looking into Eclipse IDE is not as photogenic. But in our day to day job I find myself arguing about the same things as people in this movie.

There is a question of recognizing good and bad design: knowing that you got it right, Admitting that you got it wrong. I do not care what people say but we all go by some sense of internal beauty here. You say that code is elegant or (more often) messy. You say that something feels right. That's exactly how people in the movie talk about their experience.

In some way we - coders - have it easy: we work in a medium that translates directly into the resulting product. Software compilation is handled by computers; your design is expressed in code. You work in a strictly defined - and some say limited - vocabulary, but if you follow syntax rules you do not have to worry that someone misinterprets your vision. Not so much for architects, if they do not find a proper way to express themselves through drawings, models and design documents the resulting building might not be what they imagined. They show one of such buildings in the movie: it still looks great to me (but I did not "code" it).

Architectures - like programmers - are hired help. The interesting thing is that in both cases frequent interactions with customers actually result in a better end product. It's not a one way street, but a learning process were vision shapes the expectations and expectations shape the vision. There is even a similar feeling of an establishment pressure. It's not easy to propose and build something different. Gehry's buildings are like Linux in Windows world. (And yes I have to say it: some of those buildings do not have windows - at least not in a traditional sense)

Similarly to coding, in architecture one can do very little alone. You have to have a team of people. Organizing such team, making sure that people are motivated and happy to work on the same project is an art in itself. And in architecture - as in coding - you need to have proper tools. It's kind of sad that probably architects are better than us at using computers to develop their ideas. The stuff that they do with computers is amazing. We live in the world of primitive editors, outdated version control system and sink under the code that has not been automatically tested.

If you code for living this is a film for you. Even if you do not care for this obvious 'architecture is like coding analogy', even if you will not like the amazing buildings that it shows, you will appreciate the live demo of what computers can be used for. And you will feel proud that you are in this business.

The last laptop I'll ever have?

2007-08-18T05:52:00.001-07:00

Recently I was changing laptops with a head spinning frequency. My old trusted Dell Latitude D600 decided to finally quit on me: with a whimper not with a bang. The relationship lasted 3 years so I was not really surprised. It took me about 30 seconds to put my disk in a similar laptop, which unfortunately decided to quit as well. Than I found a HP Pavilion that weights about as much as baby elephant and makes sound similar to that of the starting jet engine. Applying adjective 'portable' to this beast is probably a text book example of unintended irony. Finally I got a new HP Compaq nc8430. This wild ride made me realize couple of things.

I take it for granted now that I can take my hard drive, pop it into completely different machine and my Fedora will wake up, dutifully discover everything and will start working. I might get suboptimal video or my wireless might be missing, but everything else will just be there. And whenever I cannot take the hard drive (apparently the standard changed when I blinked) I can take my trusted home directory.

It sounds normal but when I mentioned it to my Windows based colleagues they were certain that grieving the loss of my Dell confounded my sense of reality. Everybody apparently knows that switching laptops can take weeks if not months of painful reinstalling, countless migrations application and hours on the phone with your friendly customer support transferring your software licenses. Why exactly people put up with it is beyond me.

I am proud user of nv8430 these days. Switching to HP is a company mandate. I cannot really full-heartedly recommended it for daily hacking. Screen is gorgeous, but ATI video card does not really run well unless you install proprietary driver. No Beryl fun to be had, so I cannot show off to my Windows friends. The fan never switches off and runs quiet only if I set ARI card in a power saving mode. And apparently 'business class' is marketingese for 'ugly'. But with its dual core it's noticebaly faster than Pentium M Dell. Cuts sipXconfig the compilation time to about 70%, and I can run infamous sipXconfig's 'ant precommit' target (which cleans, compiles and run all kinds of tests, including database and UI tests) in about 5 minutes (used to be more than 10 recently). The battery life is decent too: measured in hours not in minutes.
I am yet to figure out how to put it to sleep or hibernate. Out of the box Gnome's sleep option makes X to go away and text console spitting some garbage (I am blaming the proprietary video, but I have noo proof yet).

I certainly hope my hard drive changing behavior would soon become a norm. I will stop carrying laptops altogether. I will just carry my a flash disk with favorite Linux installation. There will be a powerful, generic diskless station in the office where I plug in my hard drive and after 10 second boot procedure I will have my familiar environment. And even that will be a short-lived fad because the internet will make it possible to store OS image on-line.
So this might be the last laptop I'll ever have. I should be nicer to it.

Hold the Secret Sauce please

2007-07-20T13:13:00.001-07:00

Brian Rigss wrote a small piece on Nortel and SIPFoundry. It's an interesting news for all VoIP fans and it's a very good news for SIPFoundry community.

I was slightly surprised that open source hackers now apparently roam in loose-knit bands. It is a quaint image evoking sorely missing sense of adventure. As charming images go, this one has little to do with reality and is easily dispelled by subscribing to any devel list - including sipX one - and counting the references to my boss and my company.

I cannot resist having some fun with his secret sauce metaphor. To quote Brian:

[...] VoIP software has been sacrosanct. It’s the secret sauce. It’s the way IP PBX developers ensure that their customers have access to the most advanced sets of call features. It’s how they add value to their product, differentiate from competitors, generate the majority of their revenue, and transform themselves from providers of monolithic PBX cabinetry and into communications software developers of the future.

I happen to think that Secret Sauce turned out to be not what people hoped it to be. Actually, I think that it's not a sauce at all but a trans fat., which - at least according to wikipedia - is neither required nor beneficial for health.

If I were running a corporation, I would be suspicious of my communication system being dependent on a secret sauce. What happens if I get hooked and secret sauce provider goes out of business, what if their secret sauce does not mix well with the secret sauce of the other guy. Do I really want to take this risk?

I prefer my secret sauce provider to start making money on something that I actually want to pay for: developing more features, faster and better than their competition. Without hiding behind a secret recipe.

In several US states overactive legislatures banned trans-fats. I wonder if the Secret Sauce is next on the agenda.

subversion: overdue good-bye?

2007-05-27T06:58:00.001-07:00

I discovered subversion before it got to 1.0. I convinced people working with me then to switch. At that time we were using ClearCase. Simplicity and speed of subversion was in comparison exhilarating. I even wrote couple of simple ruby scripts to bridge official ClearCase and our subversive repository. Even the name was perfect. Let's just say that even my undeniable sense of humor and personal charm failed to mend strained relations with Release Engineering group (yeah - it was a big company).

When I started working in Pingtel sipx developers already had been using subversion. Since it was about the time we needed to create our first real branch I took upon myself figuring out how to use svnmerge. We never looked back.

If that was not enough Producing Open Source Software that Douglas made me read at one point practically defined the way I think about my job. And it's written by Karl Fogel - yes - the subversion Karl Fogel.

I am writing all that to prove that I really like subversion. So it saddens me to realize that my favorite tool is most probably no longer the best tool.

As a part of my daily job I am committing patches sent by contributors. From time to time I create branches for people who want to experiment with sipx without breaking mainline. And slowly I begin to think that my job would be easier with a different tool. And that with a different tool sipXconfig and sipx would get more contributions and better contributions.

With subversion - or any other centralized SCM - there are always two categories of people: those with commit access and everybody else. Committers and patch senders. In-crowd and followers. Eloi and Morlocks.
As a patch sender you have to create your working copy, make your changes, send the patch - and - never use this working copy again. At least not until someone with commit access actually commits your patch. If the patch gets not accepted in its original form and you want to fix it, you are losing the history. You cannot somehow save your changes locally and than make changes on top of your changes. At least if you only have one patch you can keep updating the working copy and track the mainline. If you have more than a single patch however you are doomed. Subversion does not make life easy for you. I wonder how many useful contribution sipx is losing just because it's too hard to contribute anything more than a simple patch.

But even once you get commit access the problems do not stop. Before you get commit rights your every step is watched, every patch reviewed, every extra space corrected. Once you do not have to send patches for approval anymore it changes. It's not about becoming sloppy. It's about the lack of feedback. Even if someone does review your commits, and people do, once you check something in, it's in. Official. With good chances to survive forever without being looked at. One of the biggest benefits of open source, constant peer code review, is not the default option.

I started thinking about it after reviewing several patches last week. I took some. I rejected some. I sent comments or my own fixes on top of those patches for review. I knew what I was doing with subversion all the time but somehow it felt like I am working around my SCM rather than using it.

I thought of darcs which I discovered when trying out buildbot (sipx nightlies are courtesy of buildbot). That lead me to git, mercurial and others. It quickly became obvious I was not alone. Mozilla chose Mercurial over Bazaar (both distributed). They did not even shortlist subversion. Linus with his usual flair refers to subversion as the most pointless project ever started. He has a point. Open source project should never use anything but distributed SCM.

Source Control Management history follows the script leading to more freedom to modify and experiment: "lock-modify-commit" gets replaced by "copy-modify-merge-commit" (no locking) which in turn morphs into "pull-modify-commit-push" (no central repository).

If only I could figure out how this tailor thing works, I would set up mercurial mirror of sipx and encourage contributors to use it. Exciting.

Sometimes it's just better to let go

2007-05-21T06:48:00.001-07:00

Contrary to what many people think open source applications are not created by a bunch of crazy, willing to work for free volunteers. At least not only by them. Read Linus's biography to find out more. Often open source contributors are engineers employed by big and small corporations.

There are many reasons why a for-profit corporation might consider sponsoring or developing open source projects. Ensuring support for a project that is outside of company's core strategic interest and assuring the customers that they are not vendor-locked are the most commonly cited ones. Another reason - probably not so often discussed - is cooperation with your business partners.

Software companies often engage in shared deals, shared developement, strategic partnerships. It's a not-so-well kept secret that deals like that often fail, generating little more than several widely discussed press releases. It's usually the clash between corporate cultures and internal politics that prevent even reasonable deal to go forward.

The situation changes when companies cooperate within the realms of on an open source project. Especially, if it is a well established project with wide community of users and developers. First of all the rules of cooperation are commonly understood, established and transparent. The roles of sponsors, contributors, reviewers, project coordinators are well defined. It's really no that different from having a standard API interface - only these times it gets applied to people and processes not to software components.

Simple things that can take ages to establish like 'how I can get your source code' and 'what format is your documentation' are already resolved.

The biggest improvement though is in the area of day to day communication. Engineers and product managers are forced to discuss their ideas openly to get them implemented. The usual management structures that in closed source world lead to exchanges like 'tell your boss to talk to my boss before talking to me' expose their absurdity under the open source light.

No side of the deal is afraid that the other side is going to steal IP. Reciprocate licenses eliminate the false temptation of competive advantage that can be allegedly achieved by stonewalling.

This is what big bosses like best: you reduce your risk by outsourcing your communication and product management to someone who just does it better than yourself. In this case it's an open source community.

AudioCodes gateways support is here

2007-05-07T07:49:00.001-07:00

I am kind of relieved that AudioCodes gateways support achieved the stage when it's actually useful. To tell the truth I had my doubts it would ever happen. Also the more I work on those blue boxes the less I understand how one can configure them without sipXconfig. There is so many settings that have to be set in a very specific way for the entire setup to work. Simplicity clearly was not a goal here.

The good news is now you do not have to do that anymore. There is some bootp/DHCP magic that I am going display on wiki, but once you configure that it becomes a plug-and-play affair. So call you proverbial mothers to setup your VOIP networks. See wiki for more info.

RINCOR

2007-04-05T11:26:00.000-07:00

...which of course stands for "Ruby is not C or Java"

In the midst of programming sipXconfig I need to fix call resolver bug. I need to count 'setup' and 'end' for each call leg.

I write my test. I add 4 lines of code. I end up with something like this:


    @refcount--
    if @end_time.nil? || @end_time < cse.event_time
      @end_time = cse.event_time
    end

I run my test and Ruby complains:

Exception: undefined method `-@' for Thu May 17 19:30:00 UTC 1990:Time

To which I eloquently say: what???

Couple of moments later with Douglas looking over my shoulder:

Douglas: is @end_time even defined
Me: scrolling to the definition
Douglas: silence
Me: silence back

Desperate problems call for unusual measures. Let's print @end_time


   @refcount--
   p @end_time

Ruby move follows. And Ruby says: syntaxt error

Can you see it now?
It took us about 30 seconds since syntax error. Turns out @refcount-- is not how you decrement in Ruby; @refcount -= 1 is. Which we always knew. Well: kind of...

subversion 'since' trick or how to deal with short-term memory loss

2007-03-15T11:33:00.000-07:00

I guess I always kind of knew that svn log and some other subversion commands accept the date ranges instead of revision numbers. But the format is so quirky I suspect many people never use it.

If you lucky enough to use newer client you can take a wild guess and say:
svn log --limit 10
You'll get at most 10 changes this way. But if you want to find out what exactly changed since since March 1, 2007 you can say:
svn log -r {2007-03-01}:HEAD
I usually want to check the changes since yesterday (you know short term memory loss attacks and I need to find out what I am doing). So I ran Google search and guess what! There is a way to convince bash to display yesterday's date.
date -d "1 day ago"
Of course this is not what subversion likes. But we can always specify format.
date -d "1 day ago" +%Y-%m-%d
There is no way I am going to type it every time but bash and its aliases comes handy:
alias svn.since="svn log -r {\`date -d '1 day ago' +%Y-%m-%d\`}:HEAD"
This is one cryptic alias, at least to my untrained eye. What's more it's stuck on displaying yesterday's changes which does not help much good on Mondays.

Aliases in bash apparently do not like parameters, but there is another handy feature: functions. Using it is as easy as adding the following to my .bahsrc:


svn.since() {
       since=`date -d "$1 days ago" +%Y-%m-%d`
       svn log -r {$since}:HEAD
}

And now I can type:

svn.since 3

to remind myself what was going on in my favorite project over the weekend.
I'll better code and check in something now...

Why open source is so hard to use.

2007-02-28T10:51:00.000-08:00

1.) Open source projects tend to be agile; do what's needed.
2.) If your project generates revenue, you do what makes you money.
3.) Once customers know how it works, why would they pay to make it easier (unless they are in a position to have to train others.)

Each month we plan a scrum sprint, each month I tell myself to tell the product owner to select usability related items from the backlog, and each time I talk myself out of it because __this month__ there is revenue directly tied to these new features.

How does one go about addressing usability in an open source, scrum model? Should some percentage of sprint points be dedicated to usability concerns each month? We did this once and it seemed to work, but it was very hard for product owner to swallow and would be very hard for him to swallow every knowing he'd have to do it every month.

It does force a developer to be fairly creative about how to increase usability on a tight budget, which can be fun.

Sorry state of Tapestry

2007-01-07T07:23:00.000-08:00

I still remember how about two years ago lazyboy and I spend an afternoon analysing pros and cons of various Java WEB frameworks. We were mostly choosing between Java Server Faces and Tapestry. We actually had a whiteboard and assigned points to various criteria such as open source community acceptance, syntax of meta files, testability etc. If I remember correctly the major strike against JSF was its lack of open source credentials (it was when My Faces were still before the first release). In the end Tapestry won, since lazyboy wanted to use it more than I wanted to use JSF. If I ever happen to have such discussion again, I will make sure to take into consideration release frequency.

I was never sorry sipXconfig chose Tapestry. We had a major success with it. We rewrote seemingly un-rewriteable network of JSP, javascript and godknowswhat code. We reduced the code size, accelerated development, took full advantage of component reuse. It was not automatic, but we even upgraded from Tapestry 3 to Tapestry 4 on the way, since we wanted to take advantage of the new features. I always knew that Tapestry is an ingenious rather than an elegant framework. To use it effectively in non-trivial applications you have to look under the hood all the time. You have to understand its vagaries and you have to learn to like them. If you prepare to invest in such relationship, you'll be rewarded. However unless something major changes in the Tapestry governance I feel like we are nearing the end of the road.

It is the well deserved right of project major contributors to run the project the way they see it fit. They can abandon community, write a new incompatible version, stop developing: there is nothing one can complain about. However the competitive edge that open source project enjoys lies in an ongoing dialogue with the users. When you stop regularly releasing the project the conversation breaks. Developers and users are speaking past one another. Developers are touting in-progress versions that users cannot deploy, users are asking for features and fixes in versions that developers have no intention to work on. It's no accident that agile methodologies with their release early and release often philosophy fit open source project better than a venerable plan, design, code, fix, release and pray approach. They promote this precious dialogue.

If you visit Tapestry website now, you will notice that there are 4 streams of development: 3.x, 4.0.x, 4.1.x, 5.x. 3.x is an outdated legacy, 4.0.x is the official release, 5.x is a bright new world. Both 4.1.x and 5.x are under development but without any road-map in place. It's been already discussed on Tapestry list that Tapestry 5 is going to be a major departure from Tapestry 4 (just as Tapestry 4 required a non-trivial transition from Tapestry 5). If you are starting a new project you have to use 4.0. But if you do that you agree that at some unspecified point in future you'll spend energy on rewriting you project to upgrade to Tapestry 5. Even if you happy with what 4.x is offering today and you are willing to stay with it for the rest of your project life (which is always longer than expected), you cannot count on regular bug fix releases. I am not saying the bugs are not getting fixed: they are - but only in the development branches you cannot upgrade to, since those branches do not have stable releases.

Again, there is really nothing wrong with having an experimental branch. I also think it's OK to fix bugs only in the latest stream that you consider stable. But you have to have a new, usable, stable release at least twice a year. Otherwise developers, users and contributors would look for a better place to channel creative energy.

The way we are using Tapestry evolves. I changed my mind diametrically on annotations. I used to think they make code too Tapestry centric and that meta information is better left in external XML file. I do not believe that any more: Java class that represents Tapestry page or component is already heavily Tapestry centric. You do not give up much if you allow annotations there. And it is easier to code and debug 2 files (.java and .html) rather than 3 (.java, .html and .page). This is the lesson I am probably ready to take to Hibernate and maybe even Spring annotations.

The other change is a new way of binding properties and parameters contained in tapestry-prop module. It ditches evaluating OGNL and uses byte code enhancement to achieve performance that rivals that of a native Java method call. I am not sure the performance gain is actually visible on most pages, but since I never liked having too much OGNL code, I want to give it a try.

Both annotations and tapestry-prop are great examples of what Tapestry could be: not just a framework but entire ecosystem of third party components, modules, editors and various development tools. But with the current approach to the development process I doubt it will ever realizes its potential.

Code implodes for second time at 80K lines of code

2006-12-20T18:16:00.000-08:00

This line made me happy to execute today


 [dhubler@penguin main-pingtel]$ svn rm sipXconfig

For quite some time, pingtel maintained a fork of sipfoundry code, Damian and I diligently synchronized back and worth. Well Pingtel dropped it's dual license and now there's no need to keep a separate copy.

Think about it, 80K lines of merge burdening code just vanished. We haven't seen an implosion like this since sipXconfig 3.0 when we deleted legacy EJB code and went with Spring....which happened to be 80K lines of code too! I had that honor of blowing that away too. Damian deletes as much code but he prefers the old fashioned way: line by line and svn commit by commit.

So rev 8320 on sipfoundry and 7705 on pingtel of sipXconfig are identical, now I delete sipXconfig on pingtel's side and make a svn external. WHOO-HOOO!

Missing link

2006-12-08T07:22:00.000-08:00

This is not a post about evolution. Or maybe it is. sipXconfig just received a new small addition. Administrators can now easily see all the phones associated with the user. Let's say that lazyboy has a problem with his phone. Administrator can now type lazyboy in the search box, click on lazyboy user link from search results and click on Phones page to see all the lazyboy's phone. This is so much better than figuring out what is lazyboy's phone serial number and entering it in the search box. The only time you do need to know that serial number is when you are adding a new phone to the system.

There is one more bonus. If lazyboy asks for another phone (he does have 5 already, but he does talk a lot (well not really: he does not talk at all (this is kind of internal joke) (joke with a touch of Lisp syntax)))... OK let me start this sentence again. If he asks for another phone, you can just add it right from the screen displaying his existing phone. What's the bonus? Well, since sipXconfig knows you are adding phone for lazyboy, it'll add a line on that phone automatically. And you can generate phone configuration with another click; you may be tempted to never again go to the old Manage Phones list page. That is of course, if you want to spend the rest of your life managing lazyboy phones...

Why missing link? sipXconfig always allowed you to see the list of lines for each phone, now you have a link in the opposite direction. Why evolution? There is really nothing entirely new in this feature. All the components were already implemented in sipXconfig ecosystem. There was a "give all phones for this user" search, the was a phone list UI component, there were various actions like adding phones and generating configuration. All it took to implement new page was some factoring out of components and putting them together in a new way.

You may ask why I am so proud of an obvious feature. It it's really that great why did not we implement it 2 years ago. Well apparently there were other priorities 2 years ago. We implemented a whole bunch of useful features since then. Just like evolution did not start with scarlet tanager, we never intended to start with a perfectly usable system. That's what agile approach is all about: you might be missing features, even important ones, as long as you can add them quickly when they are needed. So keep on adding missing links or risk becoming one.

Big contract signed

2006-11-24T18:15:00.000-08:00

I signed a 2 year contract w/cingular for my new cell phone. My treo 600 phone quality was bad so i needed a new phone. only thing I really used on the treo was the camera phone, so i got nokia 6102i, $50 w/2 year contract. I wanted to get a mobile phone that does wifi, but I predict it will be 1yr before wifi outside home or office is remotely convienent(pardon the pun) and 4 yrs before, maybe, you can ditch the proprietary signal phones. So I'll take my cheap $40 zyxel wifi phone as a toy on trips and not buy a mixed mode phone yet, or maybe ever.

Spring into Hibernate - and it hurts

2006-11-17T09:16:00.000-08:00

It's been a long Spring debugging, google searching, forum scrubbing session. I just wanted sipXconfig to talk to CDR database in addition to SIPXCONFIG database. I thought I understood well what has to be done. New data source, new LocalSessionFactory, new HibernateInterceptor fine tuned for read only connection. I had everything wired in 10 minutes. I ran my test and spring surprised me with the following exception:

Initialization of bean failed; nested exception is org.hibernate.HibernateException: No local DataSource found for configuration - dataSource property must be set on LocalSessionFactoryBean

I double checked everything, I borrowed Douglas, which usually helps. Nothing.

Soon I was browsing in search of similar problems. I found out how to set up transaction spanning multiple sessions, but I did not need it. I found how to switch your data source based on what who is logged in, but I did not need it either. I just wanted to connect to the second database and it did not work.

Finally I found SPR-1633 which was good in a "misery loves company" kind of way. Looks like Hibernate does not really provide any easy way for Spring to inject data source, so Spring has to define a static function getConfigTimeDataSource in order for Hibernate to retrieve the data source. As soon as Hibernate is done Spring would reset configTimeDataSource. It does seem strange (or as some people call it: ingenious) but as long as you do not want to initialize two LocalSessionFactory objects in the same thread at the same time it should work.

Right. But it did not work for me. It was getting personal now: I know it's impossible to initialize 2 object at the same time from the same single thread. Everybody knows that...

Unless... Unless of course in the middle of initializing the first object, we explicitly ask the other object to initialize. Which - as it turns out - is exactly what we do.

We are way too smart to use standard Spring LocalSessionFactoryBean. We write our own extended factories. The coolest among them (the most ingenious one) is DynamicSessionFactoryBean. It extends LocalSessionFactoryBean to dynamically create Hibernate mappings for all gateway and phone plug-ins. Thanks to this trick plug-in developers do not have to modify Hibernate mapping files when adding new phone or gateway type. They do not even have to know sipXconfig uses Hibernate. In order for that to work, in the middle of its initialization phase, DynamicSessionFactoryBean asks Spring to enumerate all the beans: to find all these plugged in phones. Well, whenever you call getBeanNamesForType, Spring will initialize the remaining beans (even the lazy ones). Which of course triggers initialization of the other LocalSessionFactoryBean, which promptly resets carefully wired data source. When we try to complete initialization of DynamicSessionFactoryBean it does not have DataSource any more...

Now for the blame part. Hibernate should work better with Spring, so that Spring does not restore to ugly hacks. Spring should not restore to ugly hacks - it should find a way to influence Hibernate community to improve data source injecting. sipXconfig should find a better way to inject hibernate mappings - something that does not trigger initializing all the beans.

For now I am just going to use JDBC for CDRs. This cool lazy data source proxy that I just noticed probably cannot wait to be abused...

sipxunit?

2006-06-16T09:54:00.000-07:00

As the web layer and library layer gets more mature, we need to unit test more complex situations. For example, here is a test that I recently wanted to write: page X should be accessable to supervisor, supervisor is defined by having one or more users they supervise.

Right now, we seed our data for unit test by using webunit to drive web pages. This is very cumbersome because it can take a lot of time to write, prone to breaking (although also prone to inadvertant testing which is good), and against proper test driven principles using a layer to test the layer itself.

In addition, there's some data that simply cannot be seeded, for example results from other servers.

We knew about this limitation from the day we started, but it's unclear how to address this.

My latest thought:

I think we need to come up w/our own unit test language, library. For example I want to write this

new:user(mary) { set supervisors = new group(accounting) }
new:user(john) { groups = accounting }
system:statsServer { parameter:mockStatsServer }

By definition, this language is customized to sipxconfig, but maybe the syntax and constructs do not have to be.

Maybe spring w/it's near unlimited extensibility, and sophisticated AOP (support especically spring 2.0) will help here. Challenge with this approach is "resetting" the environment between each test. Spring has application-like events, maybe that's an approach. It's a language and constructs we already know.

Java formatting progress

2006-05-26T08:17:00.000-07:00

Java 1.4


  private String getUrl() {
      Object params = new Object[] {
          m_host, Integer.toString(m_port)
      };

      return MessageFormat.format("ldap://{0}:{1}", params);
  }

Java 1.5 argv and autoboxing...


  private String getUrl() {
      return MessageFormat.format("ldap://{0}:{1}", m_host, m_port);
  }

Java 1.5 String.format


  private String getUrl() {
      return String.format("ldap://%s:%d", m_host, m_port);
  }

you can get rid of String. using static import but I am not sure I like it


import static java.lang.String.format;
....

  private String getUrl() {
      return format("ldap://%s:%d", m_host, m_port);
  }

of course Ruby is still better:


 def get_url
   "ldap://#{@host}:#{@port}"
 end

Settings Refactoring

2006-05-11T06:16:00.000-07:00

sipXconfig can configure 6 different hardward devices and at least that many servers. There are potentially thousands of settings that can be changed. Naturally we invented a meta-model to describe these settings and how they work so we wouldn't have to code each setting in java.

Typically the problem with meta-models is that when you actually *want* to code in java for a particular because of special handling you have to deal with a cumbersome syntax.

For example, let's say I want to convert the GMT offset from minutes to hours for a phone, in java I would like to do this:

public int getGmtOffset() {
return m_zone.getGmtOffset() / 60;
}

but instead I have to do this:
Setting s = getSettings().getSetting("sntp/gmtOffset");
s.setValue(String.valueOf(m_zone.getGmtOffset() / 60));

no big deal when it's just one or two setttings, but the more and more setting you write using the meta model language, the harder it is to maintain.

So... I refactored settings so that you can write in java for most things. It relies on Java annotations, it was pretty simple. Now I'm converting all the old code, so I can simplify the system. I was pleasantly please after refactoring the code for one of the more complex devices: Polycom Soundpoint with this warning from Eclipse:

Internet in Sopot

2006-04-09T06:33:00.000-07:00

Internet w Sopocie,
originally uploaded by pirxpilot.

Spent couple of days in Poland's Tricity area (Gdańsk, Sopot, Gdynia). I was hoping for a vacation break but you know how it is. You meet your friends. Since you are a geek, most of your friends are geeks. You end up drinking beer and discussing ethernal Java vs. Ruby questions.

There are obvious perks of being in this enviable position of someone who is paid for writing open source project. The drawback is that you are perceived as a member of interesting, but also a bit creepy cult. Your friends will proceed to quickly intoxicate you and ask tricky questions such as "How can you possibly make any money on it?". So thanks Eric Raymond for your Magic Couldron or I would not have escaped alive.

For those who did not have a chance to visit Tricity - it's kind of similar to Boston. Old port, got rich on trade, very cosmopolitan place, old architecture next to shining new buildings, good restaurants, pubs filled with students and live music. Bay of Gdańsk is warmer than Bay of Boston, and windsurfing is more popular than sailing here. Poland's Cape Cod is called Hel (only one "l" here) Peninsula and Peer is Sopot is way longer than anything you can find on the East Coast but the same superiority feeling towards respectively New York and Warsaw might trick you into forgetting about 5 time zones in between.

Betas are lame

2006-03-17T20:39:00.000-08:00

Saw this and couldn't agree more with Microsoft

http://news.yahoo.com/s/zd/20060315/tc_zd/173565
-- Mary Jo Foley - Microsoft Watch

Select blubs I liked:
...

"Beta builds are predictable milestones by which external and internal testers have measured their progress toward the delivery of final code. Betas are typically few and far between, and feedback on them tends to be late and sometimes lame. But they are nonetheless a known quantity, and one that software developers, OEM partners and customers of all sizes rely on in gauging when and how to support a new product.

Open source turned the traditional software development paradigm on its head. In the open source world, testers receive frequent builds of products under development. Their recommendations and suggestions typically find their way more quickly into developing products. And the developer community is considered as important to writing quality code as are the "experts" shepherding the process."
...

"When a feature is ready, it will go into the product," Flessner explained. "I'd like to be in a situation where we can ship (a new release or update) in 120 days, due to market requirements, the team being tired or whatever."

...

The ability to install the latest CTP atop the most recent one is a priority, as well. "Customers need to be able to upgrade. When you can't upgrade from one CTP to the next, that is a definite problem," Flessner admitted.

Job openings at Pingtel

2006-01-25T19:32:00.000-08:00

Senior / Principal Software Engineer
(SIP Stack/Proxy Development)

Pingtel is looking for a senior/principal software engineer to join
an energetic and talented team to participate in exciting open-source
SIP communications servers (sipXproxy, sipXpublisher, sipXregistrar,
etc.) and presence server development projects. These servers
form the core of a SIP message routing and service infrastructure, and
are combined in a variety of ways to produce a number of different products.

Qualifications:

BSCS/CE/EE minimum
At least 7 years of commercial
software development experience
SIP, Datacom / Telecom development
experience
Solid Linux C/C++ development
experience
Experience in building and
maintaining complex multi-threaded software
Excellent software engineering
skills, including object oriented analysis and design
Able to investigate and recommend
technical solutions to complex problems with little direction
Excellent written and verbal
communication skills
Self-motivated, excellent
teamwork skills, and takes responsibility and ownership of development
areas

In addition, experience with any of the
following is highly desirable:

Experience with SIP stack
and proxy implementation
Experience with implementation
and deployment of IM and Presence protocols
Experience with XML tools
and technologies, especially XML Schema and validating parsers
Familiarity with autotools
and open source methodologies and conventions

Senior / Principal Software Engineer
(SIP-Based Application Development)

Pingtel is looking for a senior/principal software engineer to join
an energetic and talented team to participate in exciting open-source
SIP-based application development projects. The current SIP-based
applications include a SIP-based media server for voice mail, auto-attendant,
and IVR; a SIP-based ACD server; and a SIP-based Conference Server.
More exciting projects are being planned. This individual will
participate in the design and development of all existing and future
applications.

Qualifications:

BSCS/CE/EE minimum with at
least 7 years of software development experience.
Solid Linux C/C++ development
experience.
Excellent software engineering
skills, including object oriented analysis and design.
Self-motivated, excellent
teamwork skills, and takes responsibility and ownership of development
areas.
Experience in building and
maintaining complex multi-threaded software.
Knowledge of network socket
level programming and operating system primitives.
Able to investigate and recommend
technical solutions to complex problems with little direction.
Excellent written and verbal
communication skills.

In addition, experience with any of the
following is desirable:

Telecom experience, preferably
in one or more Voice-over-IP technologies (SIP, H.323, RTP, MGCP, or
Megaco).
TTS, or voice recognition.
Software-based audio processing
ACD, Call center applications
Conference bridge

A New Hope (i.e. maven2)

2006-01-03T07:41:00.000-08:00

Our build system is getting more and more complicated. It seems that we
achieved quite a lot by now:
- full unit test integration
- database and web unit tests
- checkstyle and clover (test coverage) integration
- support for Eclipse

I feel good about how easy it is build sipXconfig now (building sipX is a different story altogether). Quick and dirty start on wiki even let's you do it on without compiling the rest of sipX family. And even on the system without autoconf toolset (i.e. Windows)

There are things that are difficult still. Our Ant build scripts are a bit too complicated. You have to search for places to add new jars. It is easy to forget it in one place which usually results in problems after installation. Adding new modules is not as easy as it could be. We are slowly outgrowing neoconf/web structure but no-one seem to have the stomach to rip it apart. The dependencies are on the verge of being too difficult to manage.

However I have a feeling that we can do better and the more I look at Maven2 the more I think that this may be the way to go.

The big win is standardization. Sure you can run ant -p to find out what tasks are defined in the project but I suppose that few people will guess that ant precommit is what we usually run before svn ci, or that the ant run command has to be executed from web project to start jetty. Switching to maven would take care of these.

If a new tool that I want to use appears I want to use this tool and not figure out how to seamlessly add it to build system. And with maven there is a chance that if tool is useful there is a plugin to support it.

There are challenges of course:
- we use slightly different directory structure that the one suggested by maven - I am sure that there is a way to express it in pom somehow but I'd rather switch to standard
- we still need to place nice with make and autotools - sipXconfig is very often build as a part of entire sipX project and we do not want C/C++ developers to worry their pretty heads about our sophisticated and powerful build tools - let them stay in 20 century for couple of more years
- we would have to find out how big of a dependency problem we are introducing for various Linux platforms: of course java is a big problem already but that's hardly an excuse to make things even worse - the good thing is that we could just distribute maven and let it download all dependencies: a big plus for people who want to look at the source but not necessarily build everything - out LIB directory is probable >= 10MB and growing

Anyway - I'll keep trying. Watch this space for more updates.

Culture clash - or java programmers in ruby world

2005-12-08T07:43:00.000-08:00

If you live in a different country that you grown up in, you suddenly notice many things natives take for granted. There are of course differences in the way people dress, talk or eat. Some things are obvious: the mysterious correlation between a bad taste and programming skills among American hackers. But there is also many subtle and entertaining ways in which people behave, that provide endless conversation topics for expats. Laughing about it is a sweet revenge for all these situations in a restaurant when people are doomed to judge your IQ based on the familiarity with locally popular condiments.

In sipXconfig work I recently had an eye opening experience of a similar kind when we started using Ruby. Douglas and Walter used it to test sipXconfig SOAP interface, I got my toes wet (or should it be fingers) trying to automate importing data.

Since the novelty effect of a culture clash may wear off pretty quickly, I decided to record my experience while it's still fresh.

Well, Ruby is no Java. Ruby is like snowboarding and Java is like skiing (hence the culture clash). With Java you know where you are and you know what you cannot have. A list is a list is a list in Java and it'll be a list even after you look at this after especially intensive "coding" party. Your methods will not multiply magically. Which is comforting, but in a way sad. In Ruby you can teach a list a new skills. Classes are never closed. You can add methods, you can redefine methods. You can call methods and if they are not implemented you have a chance to do something about it in runtime. This is exhilarating and scary.

Some time ago I would not even dream about writing any serious code in a language like Ruby. I needed my compiler to catch as many problems as possible. I needed to guard my private data so that people like lazyboy do not break my beautiful code. But today I do not feel like that any more. Unit test provide me with all security I need. If it's not enough I can always write more unit test.

I also realized I became addicted to source code. Eclipse makes Java source code wonderfully accessible. Quite often I used to read library code instead of reading library documentation. Not so much for Ruby code. I am not sure if the reason for that is that Eclipse plugin for Ruby is just not good enough yet, or if maybe just too much of a library is written in C and I simply do not have source code for it. But I am a bit lost.

The good tool can make up for language real and perceived shortcomings. In Java you have to be more verbose. But that's hardly a problem if Eclipse nicely writes the most code for you. In Ruby there is a method for everything you can imagine. Finding this method name takes longer than most people would put up with.

Which brings me to the entire issue of Human vs. Minimal interface. Java users seem to be quite offended by Ruby classes with 75 methods. I kind of thing both sides are missing a point in this discussion. Culture clashes are not only entertaining. They also teach us new tricks. Java will benefit from Ruby, just a s Ruby already benefited from Java. And I am looking forward to using both.

Embedding ANT as automatic DB schema upgrader

2005-11-23T19:12:00.000-08:00

Anyone who has developed code with other developers that uses a database can sympathize with this dialog
Joe : "Hey, just checked in some code, you need to run this SQL script to create the new tables."
Mary: "It's not working, my schema already has some of those tables from last time"
Joe: "Drop all those, maybe you better drop entire db to be safe"
Mary: "But I'm in the middle of developing something, dang I shouldn't have updated my code"

sipXconfig went to use ANT to break up these little DB schema upgrades into "patches" that are intelligent enough to know when to run and remember when they've been run before. I'm very happy with this and it actually required ZERO java coding!
http://scm.sipfoundry.org/rep/sipX/main/sipXconfig/neoconf/etc/database/database.xml

Rule was you never edited a patch once it was checked in, you only patched other patches. It was still difficult in early development stages when your portion of the schema changed drastically and you left a long string of patches behind. So I came up with a patch that clears itself and applies itself everytime the application runs.


<target name="drop-firmware" if="firmware">
    <sipx-sql>
      drop table firmware;
      drop sequence firmware_seq;
      delete from patch where name = 'firmware';
</sipx-sql>
</target>

<target name="firmware" depends="drop-firmware">
    <db-patch patch="firmware">
</target>

sipx-sql runs a SQL script and db-patch does the same but also marks that the script has been applied before so that next time this ant file is run "if=firmare" will be true. Details are in the database.xml from above.

At some point when these tables stablize, I can just change it to this


<target name="firmware" unless="firmware">
    <db-patch patch="firmware">
</target>

Waiting for remote video

2005-11-17T18:10:00.000-08:00

somebody w/video call me...

Compusa had a sale on webcams, 2 Creative labs Live web cams for $70. Other one will be xmas gift so someone will call me, maybe my brother will use it.

Please, all sales final

2005-11-16T12:50:00.000-08:00

It's how I remember what order to put my java method and field modifiers

public abstract static final ...

I say it to myself half-dozen times a day, so I thought I'd share.

SOAP Authentication

2005-11-09T20:45:00.000-08:00

Decisions, decisions

As a web services rookie, authentication is a challenging assignment. First question: whether to use HTTP auth at the transport level, or WS-Security at the message level. Decision: use HTTP auth. It is simpler, mature and widely supported, and meets our current needs well.

Given the four types of HTTP auth:

HTTP Basic

HTTP Digest

HTTPS Client

Form-Based

which should we use?

We're using HTTP/SSL as the transport since sipXconfig data is the customer's private information and should be protected on the wire. So HTTP Digest auth offers no advantages over HTTP Basic. HTTPS Client auth, a.k.a. "client certificate authentication" has the extra security afforded by PKI integration, as well as extra complexity. Form-Based auth is interactive and therefore not suitable for web services.

Decision: start with HTTP Basic and support HTTPS Client later.

Implementing HTTP Basic authentication with Jetty

Jetty, the web container used by sipXconfig, is a great open source product, but the documentation is a bit scanty.

Step 1: Add security-related deployment descriptors to web.xml:


 <security-constraint>
   <web-resource-collection>
     <web-resource-name>SipXconfig Web Services</web-resource-name>
     <url-pattern>/services/*</url-pattern>
   </web-resource-collection>
   <auth-constraint>
     <role-name>admin</role-name>
   </auth-constraint>
 </security-constraint>

 <!-- Use HTTP Basic authentication. -->
 <login-config>
   <auth-method>BASIC</auth-method>
   <realm-name>jetty realm</realm-name>
 </login-config>

 <security-role>
   <role-name>admin</role-name>
 </security-role>

BEA has a good explanation of what goes in web.xml. Briefly,

The security-constraint above protects all the URLs associated with web services, declaring that only clients with the admin role have access.

The login-config element declares that HTTP Basic authentication will be used.

The security-role element declares the admin security role.

Step 2: Now how do we change the sipXconfig Java code to make the web application support this properly? Turns out that this is completely web container-dependent, sad to say, and the Jetty documentation offers little advice. By talking to my friend Damian, grepping around the Jetty distribution, and looking at the sipXconfig 2.8 code, I figured it out.

You have to create a class that implements the UserRealm interface and add it to your Jetty Server, as in the following new line in the sipXconfig JettyTestSetup class:


    m_server.addWebApplication("/sipxconfig", war);
    m_server.addRealm(new JettyUserRealm());
    m_server.start();

Here JettyUserRealm is that class. If you now run sipXconfig in test mode and browse to the web services URL, http://localhost:9999/sipxconfig/services, the browser will prompt you to log in. Of course, this is just for testing, real use of the web services API is from a client application, not a user through a browser. The JettyUserRealm test class is just a dummy implementation, real authentication via the password tokens stored in the database is the next step in our plan.

Document-centric SOAP

2005-10-21T10:12:00.000-07:00

I'm finding a lot of notes about SOAP like this
http://searchwebservices.techtarget.com/qna/0,289202,sid26_gci1098415,00.html?bucket=NEWS

In encouraging developers to think of Web service development as no more than development of a Java class with annotations, an RPC-centric approach does not encourage good service architecture, nor is interface stability likely to be maintained. The lure of a familiar paradigm attracts developers down this route, but ultimately that familiarity is an illusion: A Web service is fundamentally not like an object instance that might throw RemoteExceptions every now and then.

I need to do some more research and blows some of my illusions of grandure.

Broke ground w/SOAP, found Rubies and still not happy

2005-10-15T17:31:00.000-07:00

Being new to SOAP, XFire and Ruby, I finally got my first SOAP call the way I wanted it:
- No SOAP specific code
- No getting dirty w/writing WSDL or reading/writing XML the wire
- Unit tests that are quick and easy to write

Java Service Implementation:

So I have a Service


class UserServiceImpl implements UserService {
    public void createUser(String username, String pin) {
        // do it
    }
}

Unit test in ruby:


class UserServiceTestApi < Test::Unit::TestCase
  def setup
    @userService = SOAP::WSDLDriverFactory.new(SERVICE_WSDL).create_rpc_driver
  end

  def test_createUser
    @userService.createUser(:in0 => 'testApiUser', :in1=> '1234')
  end
end

Wow, that's hot.

But I'm still not happy. Now I have to start passing objects: user objects, phone objects. And object graphs would be nice: users and their phones. SOAP can handle objects, that's why you'd use SOAP in the first place over XML-RPC or the like. I've seen XFire automatically describe objects I use in my API in the WSDL, but how does it do the object graph. Collections of other objects. Xfire can do all this, but I'll need to understand more here. This is where I fear the community arguments like "SOAP is not pure RPC, it's a way to send XML back and forth" come into play. I'll be fighting the WSDL and giving up 1 XML element at a time.

On a different note, up until now we've been writing a web site. I've been happy with the natural layers of abstraction :

[HTML Templates] + [Business Objects]
[Web Components] + [Business Objects]
[Data Access Objects] + [Business Objects]

Notice [Business Objects] live thru all layers

So a natural copy would be

[Web Client] + [Business Objects]
[Web Service] + [Business Objects]
[Data Access Objects] + [Business Objects]

Two problems with this:
1.) [Business Objects] contain code, that doesn't get sent over the wire. They turn into dumb bags of data, linking to other dumb bags of data.
2.) [Web Service] ends up duplicating every method in the [Data Access Objects] layer and end up being dumb data movers. Anying good in the [Web Service] layer you could argue would be more reusable in the [Data Access Object] layer.

This is what I want:
[Web Client] + [Business Data]
[Data Access Objects - Stuff] + [Business Data]
[Data Access Objects] + [Business Objects]

So I want to stip off methods in DAO that shouldn't be accessable via web service, and I need to strip business objects of all functionality, typically non-accessor methods so it's just a bag of properties.

I just have to write code that unwrites code.

The Data Access Objects, Xfire already does this via annotations, let's you take an existing DAO and mark methods as @WebMethods. Great, now I have to figure out how to describe the objects. Annotations seem like a good way, but hard part would be getting inside xfire wsdl and xml marshalling code to translate.

This is a lot of design in new areas for me and w/o a lot of examples, so I'm going to trudge thru a few DAOs and see how it goes.

VoIP Developer conference update

2005-08-03T21:01:00.000-07:00

This week I'm attending the VoIP developer conference in San Francisco. Over all, a great show. 1000 people are supposedly attending.

I gave a talk about sipXconfig. About 30 people showed up. I want to thank Scott and Martin for pluging the session earlier in the day as much as they did.

Questions that came up after the session were:

- Question about running sipXconfig as stand-alone. I said there was no packaging as such right now, but that it would be possible very easily.
- Alan Percey from AudioCodes said firmware was important so bootp would be nec.
- Question about running TFTP server on different machine. I said you could use NFS, or write a TFTP service bean that used TFTP's write mechanism to send profiles
- Concern that unsolicted sip NOTIFY to signal phone restart required a proxy. (This can be resolved if we knew the IP address of the phones which is not easy on a brand new phone out of the box. If solved this would remove restriction that a phone needs to be registered inorder to restart but finding the ip address.)

The keynote from one of the large PBX vendors mentioned how they were going to a web services model to give a higher level API to handling call control. So you could issue a SOAP command to call someone. Obvisously the call doesn't originate from your phone, but if it calls your phone and the other endpoint you wish to talk to, then it's kinda interesting.

Right now we're using SOAP to configure the system, but this is where I see our web services direction going. Leverage the SIP stack to build a business applications api.

sipXconfig code implosion

2005-07-16T19:37:00.000-07:00

After 1 year and 1 month, sipXconfig developers are almost done rewriting entire application from the ground up(rewrite was not main focus, many features were added in this time). Because all the new code was written parallel to legacy code, there was an implosion last week when all the legacy code was deleted and all that was left was the new code. I had to accumulate some source code metrics to capture this effect.

This shows LOC (Lines Of Code) for sipXconfig Version 2.8 v.s. Upcoming 3.0 Version. This does not include unittest code, I'm hoping to post code coverage metrics if I get some time. We've reduced codebase by 75%. We still have a number of features to re implement, but i do not predict this number changing that dramatically.

Lines Of Source Code

it's the aspect, stupid!

2005-07-16T17:52:00.000-07:00

My infatuation with Spring is official. My significant other knows. We have an understanding. (Which is a lot of fun, since it makes my fellow programmers quite jealous.)

It turns out that the way to coder's heart goes through aspect oriented programming. This great technique, let's you solve one problem at a time. And Spring empowers you to use it.

Let's say you have this lengthy operation that should be done asynchronously. In sipXconfig world it usually means replication and until recently no-one dared to speak this term aloud without paying tribute to programming gods. By I digress...

I get to work. I write the tests. I write the code. After couple of iterations things start working. And now for something completely new. I cannot possibly run replication - which involves opening HTTP connection to sipX servers - in the same thread that builds WEB pages. The last thing our users need is extra coffee breaks, while they wait for page refresh. So I dig out my Java and threads books and start worrying... And then it hits me. The fact that my code runs in a different thread in not a property of my code. It's not even the property of the code that calls my code. It's a different ASPECT (and when I say ASPECT you need to visualize it as a fifth spatial dimension in the coding time-space). I repeat this couple of times in my head. Then I repeat this couple of times to lazyboy, who - as usual - looks unfazed. Just in case he might want to say something, I pretend that I am really busy and I run back to my computer.

It turns out Spring lets me to write little pieces of code called interceptors. I can intercept any method call and do something extra. For example spin the thread, or even better tell a thread pool to do something for me.

The callers do not see anything different. They call the functions as they always did. Just by changing Spring beans configuration I can change all my beans to asynchronous. Aspect of replicating data is isolated from aspect of doing things in the background. And you code, test and implement one aspect at a time.

This experience comes handy couple of days later when I integrate Apache XML/RPC client. This is how sipXconfig will communicate with sipX servers in future. But I do not want to expose this fact in every sipXconfig corner. I do not want to write entire isolation layer of code either. Turns out I do not have to. Again thanks to the ASPECT. Just as calling stuff asynchronously, calling stuff through XML/RPC can be handled by Invoker. Good Spring people do not have XML/RPC invoker, but writing one turns out to be trivial.

And again - the code that calls my beans does not even know that implementation of the interface lives on sipX server. Xmlrpc-iness is magically injected in the configuration file.

may be it is rocket science after all

2005-07-16T17:02:00.000-07:00

It's 2005 and you still cannot spend weekends on the Asteroid Belt. Every geek I know is seriously depressed by that. Instead, bunch of smart people is spending weekend debugging stupid fuel indicator on Discovery.

That made me think about unit testing. Pretty much eveything does these days. Some time ago I have read Columbia Last Flight. By now everyone knows about that fatal piece of foam that smashed into Columbia's wing during lift off. Apparently for a long time, despite of increasing evidence, some NASA engineers could not accept that. They decided to test the "foam did it" hypothesis. They were actually shooting foam pieces into shuttle panels and monitoring the damage. William Langewiesche in his Atlantic Monthly article describes the critical test: The gun fired, and the foam hit the panel at a 25-degree relative angle at about 500 mph. Immediately afterward an audible gasp went through the crowd. The foam had knocked a hole in the RCC large enough to allow people to put their heads through.

I like to think that this is what happens when you type "ant test-all". A gun is fired, big piece of foam hits sipXconfig code, damage is assesed. There are no gasps, just test report. And if there is a hole, no-one's life depends on it. (Well, there is this e-mail from lazyboy with your morning coffee.)

Now the sick part: it almost makes the lack of Asteroid Belt vacation package bearable.

resonance

2005-06-25T07:52:00.000-07:00

Contributing to sipXconfig is one of the most exciting things in my career as a software developer. I blame “open source resonance” for that. We programmers are a lazy bunch. It’s a good thing too. I did work in the past with coders that were not lazy and could write code faster than I could delete it – what a pain!

And, since we are lazy, we hate doing stuff over and over again. This is what computers are for. People are for pressing buttons and enjoying a nice glass of their favorite non-alcoholic beverage on quiet evenings on the shore of Lake Geneva (not that I ever enjoyed a quiet evening there, but I suppose every lake deserves a mention in our blog from time to time).

Here’s the impressive list of libraries and tools that sipXconfig is benefiting from. I cannot possibly enumerate all, but I’ll try:

The “big three”:
- Spring – a new plumbing of sipXconfig – declarative dependency manager that behind the scenes performs a magical incantations on our humble Java objects
- Hibernate – crosses the digital divide between object universe of Java and table universe of databases
- Tapestry – encapsulates weird WEB protocols and technologies into cute reusable components meaning there is a chance we will finally have a consistent UI and will be able to forget about copy and paste option in our text editors

Silent friend:
Eclipse – our development environment with refactoring support, amazing code browsing capabilities, supernatural auto-completion, and last but not least a satisfied green bar of all test passed. Everyday is discovery, every versions brings a new set of goodies meaning that more can be done in less keystrokes. Suddenly your typing skills are less important than the problem you are attempting to solve.

The risk enablers:
JUnit and family – DBUnit, XMLUnit, WebUnit, HTTPUnit etc. - these guys are responsible for the fact that we can add features and fix bugs till the very late in the release cycle, they ensure that 95% of our nightly builds are actually quite usable, they empower bold architecture changes, they ban code areas that no-one wants to touch because the last person who understood how they worked died in the Great Fire of London; they improve our design, and - if we do stupid things - they leave light out there so that we know how to safely back off

Also all jakarta libraries, checkstyle, dom4j, xerces, digester, postgres, jetty, tomcat, cglib, ognl, subversion: none posible without open source. Open source friendly bug track and test coverage tool (JIRA and clover) deserve an honorable mention too.

So we are indebted: and that’s why we post from time on Tapestry or Spring list showing others how we solved some problems. (Well that, and the fact that we are terrible show-offs and attention seekers.)

And that’s why we hope that more and more people will want to contribute to sipXconfig – filing bugs, implementing new phones or gateways, writing wiki notes. We want to get to the point the resonance works for us too. And when we get there, you’ll be able to manage your toaster with sipXconfig. That reminds me of something. I wonder what is this strange smell from the kitchen....

Where Were You When sipXconfig Hit 1000

2005-06-08T11:21:00.000-07:00


Transmitting file data ......
Committed revision 991.

Transmitting file data ......
Committed revision 992.

Where were you when sipXconfig hit it's 1000th checkin? Well it hasn't happened yet, but like Dick Clark's reflection on the year leading up to New Year's Day, it's time to reflect on all the special events that have happened to sipXconfig in it's life inside subversion.

And so it started 11 months, 343 days ago.


r1 | bob | 2004-06-21 12:01:50 -0400 (Mon, 21 Jun 2004) | 3 lines

initial check-in

Damian joins the fray.


r86 | dkrzemin | 2004-10-18 15:37:29 -0400 (Mon, 18 Oct 2004) | 4 lines

XCF-40 fix:
- subant redefinition pulled out of subant-defs target and replaced with my.subant definition
- subant-defs target removed

Checkstyle was added, what a beautiful thing.


r150 | dkrzemin | 2004-11-17 19:39:32 -0500 (Wed, 17 Nov 2004) | 1 line

Initial support for running checkstyle during build added.

Immediately after, Neoconf what is now half way to becoming complete replacement of all previously existing EJB code, then just a gleam in my eye.


r151 | lazyboy | 2004-11-18 11:09:42 -0500 (Thu, 18 Nov 2004) | 1 line

Add Neoconf module (at this time optional see neoconf in manual) the evolution revolution! 
Add hooks to turn debugging on thru env. var.

Tapestry introduced, what is also now half way to becoming complete replacement of all JSP pages.


r255 | lazyboy | 2004-12-15 18:15:52 -0500 (Wed, 15 Dec 2004) | 1 line

Include some experimental tapestry code - coexists with JSF

JBoss is upgraded from 2.4.4, to 4.0. The "Mount Everest" of challenges Damian met face-to-face! This was the first on ongoing ripples in the JBoss upgrade pond.


r228 | dkrzemin | 2004-12-10 20:05:57 -0500 (Fri, 10 Dec 2004) | 1 line

JBoss4 upgrade: unused tags referencing removed classes deleted

HTTP Unit tests were added, another beautiful thing.


r318 | dkrzemin | 2005-01-06 19:11:41 -0500 (Thu, 06 Jan 2005) | 2 lines

httpunit tests added - for now can be run from Eclipse only

Damian he's so honest...I would try to claim I was fixing a bug.


r569 | dkrzemin | 2005-02-28 14:02:20 -0500 (Mon, 28 Feb 2005) | 2 lines

undoing erroneous commit

On our way to 3.0 release, best one yet.


r626 | ajorde | 2005-03-14 10:30:45 -0500 (Mon, 14 Mar 2005) | 1 line

Bumping version up to 2.9.0

Code contribution from Hannu Strang, sipXconfig's first from community.


r838 | lazyboy | 2005-04-30 01:57:08 -0400 (Sat, 30 Apr 2005) | 1 line

XCF-306 Initial support for Cisco 7960, 7940, 7912, 7905 and ata18x devices in neoconf library. 
Integrated contribution from Hannu Strang.  Still a lot of work to do to replace support for
cisco devices in old model including group support.  Stilll need to refactor to reuse code
between polycom, add encryption support and fix settings losing order

Run in with the law.


r850 | dkrzemin | 2005-05-03 16:04:41 -0400 (Tue, 03 May 2005) | 1 line

subversion police: unix specific files should have LF as a newline characterr

Remove nasty requirement to install generated certificate authority into Java's VM.


r951 | lazyboy | 2005-05-31 15:52:16 -0400 (Tue, 31 May 2005) | 1 line

XCF-397 - Separate truststore into separate file

Damian finds ingenious way to configure hunt groups using aliases and flexible SIP messaging techniques


r983 | dkrzemin | 2005-06-06 19:22:05 -0400 (Mon, 06 Jun 2005) | 1 line

XCF-343 - generate aliases with sipx-noroute parameter

Our baby needs a type system

2005-05-21T05:35:00.000-07:00

sipXconfig is growing up. And looks like it needs a setting type system. It's a frightening moment in the lives of proud parents. The teething problems are over (meaning we have settings), and it's time to start saving for college. The panic starts: which bank (which types we support), which tax deductible option (XML mark-up anyone?) and finally: where to take the money from (we do have so many other things to work on).

OK. Enough with the lousy metaphor...

The problem at hand: all our settings are strings. And I do not mean internal representation only. We treat them as strings across the board. Which makes it really cumbersome to implement any type of input validation or custom renderers. The natural solution would be to allow model designers to specify the type of the settings and the constraint of this type in the XML file. The problem is there is so many options to choose from. We want flexibility and extensibility: user defined enumerations that can be reused for multiple settings. But most importantly we want simplicity. We certainly do not want to roll out a new "type markup" language. One can very easily go overboard with that - just look at XML schema.

So I am thinking we start small and see where it takes us:

- number (integer) with min and max constraints
- string with regular expression constraint
- string with password constraint
- enumeration with labels and values

Every type will have a preferred renderer - in case of Tapestry widgets that would be NumericField, ValidField and PropertySelection. With time we figure out how to allow for custom renderers, user defined types etc.

This should keep us occupied for a while...

Ajax and Tapestry

2005-05-03T17:40:00.000-07:00

Looks like Ajax and similar technologies (partial page rendering, micro posts and other kind of magic) are becoming more and more popular. They look cool, they are fast, and at least when one uses proper technology they are not difficult to implement.

I was a bit envious about Ajax components implemented in the world of JSP and Java Server Faces, but I also found this.

Partial Form and Partial Rendering is exactly what I needed for some time. I am going to try it.

Folders v.s. Tags

2005-04-12T19:59:00.000-07:00

Flickr, Del.icio.us and gmail use a concept called "tags" or "labels" to organize images, bookmarks and email respectively. Create tags as you go, items can have mulitple tags giving you a cross index. Not a new concept, but an "advanced" one, for the general public.

sipXconfig uses the classic Folders everyone understands, but there are a couple of limitations I hate. You create folders first then add items. Items can only be in one folder.

So in sipXconfig's situation, if a admin is trying to manage 10,000 users and their phones clearly they need two things above all else:
Organize items so even someone who didn't setup the system can find things (drill down)
Allow admins to share settings between the items.

The only customer in the thousand item range has done what most admins would do, create 100 folders in preparation for settings that may need to be different. Like single-heirarchy systems, it's relatively straightforward, if not cumbersome, to find items by drilling thru nested folders, but it's hard to change how settings are shared between folders without major restructuring. In addition, restructuring may be in conflict with goal to organize items so they're easy to find.

sipXconfig as Tags provides a lot of flexiblity when applying settings. An admin can add phones to the system and tag them with as many qualifiers that match their need at the time Example tags: "sales experimental" or "downstairs lobby 340-ata" or "engineering george". Physical location, models, user or users audience.

Million dollar questions: if search is good enough, do you have provide users the ability to "drill-down" through items and would it confuse them that items may show up in multiple places. Is this a user issue or a presentation issue? Also, I hesitate using the word "tag" and would like to use "folder" but this may be trying to pass an old dog as a new pony and confuse everyone.

Writing out load, do both. Use folders, first, as a organizational mechanism. Folder names are unique across the system, items can only be in one folder. If an admin wants to attach additional "tags" to an item, they can do that w/o affecting it's folder heirarchy. Kinda like ruby language, single inheritence but has mixins. Best of both worlds, but not clear how to convey this concept even to the advanced admins. Of course, in agile terms, sipXconfig could just support folders for now but design in a way to leave opportunity for "tags" if/when there's a demand.

XFire SOAP support is nice

2005-04-08T13:28:00.000-07:00

Currently we're using Axis SOAP that comes w/JBOSS. We're moving away from JBoss and all the code written in JBoss so it's as good as time as any to seek alternatives. Using Spring has made me lazy. Create classes and interfaces for what I do and everything else is just configuration of what to do w/those interfaces. So XFire is all about that too, give it the interfaces and let it register it as SOAP service, generated the WSDL, and even generate the client stubs which I plan to use for testing right now. Real SOAP clients are in C++ and perl. I hope XFire plays nice with C's csoap library implementation and Perl's SOAP::Lite.

So in my business library project, neoconf, I don't have any SOAP library dependencies

interface MyService {
void String basicMethodWithBasicParams(String basic, int basicToo);
}

and it's not until my web project that I expose this interface in web.xml and myservice-xfire.xml and that contains no code.

NICE!

Took some doing to get XFire to build. Followed directions on website w/a little help from mailing list about maven issues, of which I know little about.

I am a bit worried about all the dependencies:
file="${wsdl4j.jar}"
file="${wsdl4j-qname.jar}"
file="${stax.jar}"
file="${stax-api.jar}"
file="${javamail.jar}"
file="${activation.jar}"
file="${yom.jar}"

for client
file="${xbean.jar}"
file="${velocity.jar}"

But worth the price, and can't complain I'm sure it make the library very capable by leveraging other libraries.

JBOSS in anger, years after

2005-04-04T19:03:00.000-07:00

Starting with sipXconfig 2.8, progess has been made towards porting existing codebase from JBoss and JSPs to Spring Framework, Tapestry and Hibernate (TapSprHib). It's not that JBoss is an inferior technology, it just doesn't meet sipXconfig requirements anymore. Here my current comparison
- Memory requirements
o 400MB RAM with JBoss - JBoss 4.0.1 currently starts at 320MB and can grow to 400MB shortly after running. It can grow even more depending on what you're doing
o 60MB RAM with TapSprHib, this will grow as more is code is ported and libraries evolve but should stay in 100MB ballpark
- Startup times
o 1 minute, 57 secs with JBoss. This has already come down from even longer startup times by disabling many services. It's possible a few more could be disabled
o 11 seconds with TapSprHib
- Installation/Deployment
o installing jboss from tarball requires many manual steps including:
- patching JBoss installation
- copying in certs.
- setting file system permissions properly
Binary (e.g. RPM) installation is easier but this has other issues
- maintenance of custom RPM for SIPfoundry
- availability on other distributions is limited or behind in versions
- either limits development to work on 1 copy at a time or complicates
develop environment
o none - embedded Jetty jar file with 1 embedded configuration file
- Download/HD requirements
- 60MB+ as separate download for JBoss
- 4MB as part of sipXconfig download. This is a total of Jetty, Tapestry and Spring jars. Hibernate not included because its required for both

From these numbers, JBOSS is better suited for high-end enterprise systems, not as a transparent, integrated, web admin tool. EJB 3.0 spec and subsequently JBoss is evolving in an encouraging direction but it may be too little and it's certainly too late for sipXconfig.
Damian and I have already ported roughly 20% when refactoring daling plans, gateway management, managed polycom phone support. Damian and I will take a similiar tactic for sipX 3.0. Unfortunately users won't see the fruits of our labor until JBoss can be removed entirely. I logged the remaining portions of the port in the sipXconfig bug database as tasks: XCF-293 through XCF-306. I roughly estimated 60 man days to port remaining portions.

FireFox for WEB developement

2005-04-01T07:27:00.000-08:00

WebDeveloper is an extremely useful FF extensions that help with WEB pages design, reverse engineering and specifically with testing CSS.

My favorite feature: you can get click on any item on the screen and see all CSS styling that influence its apperance (CSS->View Style Information)

You can hide the toolbar if you happen to use FF just for browsing.

reasons not to overuse XSLT

2005-03-28T12:53:00.000-08:00

I fixed a bug.

This is why I dislike overusing XSLT. We take the data from couple of different sources, cram it into XML document and generate HTML page through XSLT transformation. There are at least two really bad things here:

We are passing data between different application components as XLOBs (XML Large Objects (tm) ). I do not care about how ineffective it is, I do care about how non-transparent it is. You never know what someone decided to put into that XML and you never know who and what depends on its content.
We are using XSLT to apply bussines logic. In this case to decide which components are editable and which are read only. Aa a result our UI code is hopelessly hidden in the ocean of bussiness logic writen in poorly designed and not especially convenient programming language (XSLT).

The bottom line is template based approach (JSP, Velocity, XSLT) quite often degenerates in the heavy proprietary scripring, where template stops resembling your destination document (which is the main reason to use the template after all). I think I prefer Tapestry to JSP, DOM4J/JDOM/Xstream to Velocity, Java/Python/Groovy to XSLT. Also I prefer cats.

Tapestry components repositories

2005-03-24T11:48:00.000-08:00

Since Tapestry related tools are using a lot of real world terminology it's virtually impossible to look for anything Tapestry related on google (try Tapestry, Spindle, Tassel and see what you find).

This is a list of Tapestry components repositories that I know of:

Tapestry in anger: pitfall #12141

2005-03-17T14:03:00.000-08:00

You cannot validate your page when the form tag is in an embedded component because the IValidationDelegate.hasErrors() will not process the components form elements until it's too late.

So mypage.page (pseudo code)

mycomponent
submit button [w/listener=page.doit]

mycomponent.jwc

form tag
form element w/validator defined

MyPage.java

doIt() {
IValidationDelegate delegate = (IValidationDelegate) getBeans().getBean("validator");
// INVALID: form elements in component has not asked to validate yet!
boolean save = !delegate.getHasErrors();
}

So I rewrote this so the form tag is in the page and now it works fine.

Tapestry: accessing validators from components

2005-03-17T11:12:00.000-08:00

binding name="validator" expression="page.global.validXXX"

This works for accessing all global data from components, use judicisouly, as components make assumptions about page they're in

logging according to lazyboy

2005-03-11T07:42:00.000-08:00

Apparently this is how one should use commons logging in sipXconfig:


// define logger as private static final field
private static final Log LOG = LogFactory.getLog(SipServiceImpl.class);

// use it
LOG.info("blah");

The output goes to server.log

Damian's flowers

2005-03-09T19:50:00.000-08:00

Strategy for binding Interface implementations from Spring with Hibernate object

2005-03-09T17:48:00.000-08:00

This is a very esoteric blog, it's more of a mental memory dump for myself incase I ever need to recall this "design pattern"

Actors:
1. Phone interface
2. Vendor's Phone interface implementation
3. PhoneData hibernate object w/access to perisent data for all phones
4. Line interface
5. Vendor's Line interface implementation
6. LineData hibernate object w/access to perisent data for all lines

Goals:
1. Let interface be service driven, easy to implement
2. Let database store simple properties and not burden implementation with menial getter and setters and allow interface and persistent objects to evolve independently of each other

Challenge:
Phones have 1-to-many relationship with lines and hibernate is great at dealing w/that however hiding data objects at least when it comes to adding and deleting lines becomes very difficult.

Knowns:
Dao needs to wrap each read/write of data objects to return interface implentations. Therefore fancy queries that return complex object graphs are not really useful.

Solution:
Given that all queries need to be "wrapped" advantage of having hibernate manage the collection is minimal. Therefore, if phone interface manages a "simple" collection, Dao can read/write collection in only a few isolated places. I say "simple" because one trick is being able to recall what lines need to be deleted. A simple Collection subclass handled this trick.

hibernate: deleting objects w/unsaved references

2005-03-09T13:40:00.000-08:00

Hibernate Gotcha #104312

Error: object references an unsaved transient instance - save the transient instance before flushing

Senario: I was deleting an object that had a reference to an unsaved object. Why would i try to add a new object to an object that's about to be deleted? Improper logic, but, not exactly illegal. Now I null out the reference and it's happy.

DBUnit seed data, use high primary ids

2005-03-09T11:15:00.000-08:00

Using dbunit w/postgres, or any db that uses sequence numbers, is challenging. I just spent half a day finding out why seed data should contain primary keys starting at high numbers, say 1000.

Senario:
1.) create new db
2.) insert a new row with primary key = 1 in dbunit seed file. NOTE: Dbunit does not update sequence numbers. (if this scares you, it should, almost ditched it thinking it was going to be useless, but for some reason I decided this could be worked around)
3.) write a test that adds a new object from db business layer, if this is your first unittest, it will pick primary key = 1 as well and will throw an error. You would think the error would be obvisous, but I didn't "get it" from this hibernate/spring but from lot of debugging:

org.springframework.orm.hibernate.HibernateSystemException:
a different object with the same identifier value was
already associated withthe session: 1,of class:
org.sipfoundry.sipxconfig.phone.LineMetaData; nested
exception is net.sf.hibernate.NonUniqueObjectException:
a different object with the same identifier value was
already associated with the session: 1, of class:
org.sipfoundry.sipxconfig.phone.LineMetaData
net.sf.hibernate.NonUniqueObjectException: a different
object with the same identifier value was already
associated with the session: 1, of class:
org.sipfoundry.sipxconfig.phone.LineMetaData

Spring's AOP Major limitation

2005-03-05T02:53:00.000-08:00

Apparently Spring with only affect bean classes it loaded thru it's application context files. Considering it weaves code at runtime, this makes sense. Knowing this, it can still be useful, but it's not in the same league as AspectJ.

So using Spring's AOP to dump particular Hibernate exceptions that wrap SQL exceptions in unittests won't be that straigtforward, I'd have to wrap each DAO with yet another proxy layer. It won't be long before people start talking about the "AOP Proxy Stack".

Tapestry contrib:Table without paging

2005-03-03T16:01:00.000-08:00

Turns out the default page size of the table component is 10. It is controlled by pageSize parameter. I could not locate any documentation on how to turn off paging altogether. In the code it looks like any negative value should result in rendering only one (possible long) page.

I am not sure if we should increase the default to some reasonable value (25, 50?) or just set it to -1 on the tables that we pretty much know are not going to have many entries (Gateways, Dial Plans).

I do not think there is a way to change the default globally (it's a static constant). So it looks like we are have to touch the code on all the pages...

autowire mode for spring beans and unit testing

2005-03-01T15:57:00.000-08:00

I was playing with several ways of setting up 2 slightly different Spring application context. One for Unit testing, another for real application. I guess classical way of doing it is to split bean factory xml into 3 parts - one that is common, the second that is used in unit test only, and the third one used in application only.
However in some cases when application context for unit test can be a subset of real application context, one can use autowire trick. Setting bean autowire attribute to byType will cause Spring to locate the singletons of a given type to initialize bean properties automatically. If there is more than one candidate Spring will complain. However if there are none, no error is reported and the property remains uninitialized.
We could employ this trick by defining "application only" beans (for example beans than need JNDI) in a separate file and include this file only in real application context. Unit test application context will not find it and as a result some bean properties in common files remain unset (null). You can either mock them, or ignore the fact that they are null if they are not needed.
I have some misgivings about using autowire mode in general, but something has to be said about more concise bean definition. For example one does not have to explicitly initialize hibernate session in each DAO.

Latest thinking on hibernate session

2005-02-25T05:24:00.000-08:00

Spring's Hibernate interceptors and FlushingSpringSessionInViewFilter are really incompatible. According to Hibernate manual "9.4.1 Updating in the same Session" and "9.4.2 Updating detached objects" UI's are more inclind to open/close session at DAO interface thus requiring save and flush methods. The FlushingSpringSessionInViewFilter would allow you to avoid having to write save/update methods which is cool in theory, but forms ought-not edit real business objects attached to session because they will get peristed on every form submit! Even worse a harmless query can trigger an automatic save (See javadoc on FLUSH_AUTO, I tested this and it does as it says).

Alternative include delegating every form input to business methods only when there's a ok/apply is maintenance nightmare as can be seen in PetStore sample app. Manually dettaching and attaching objects from session when a save is required is an approach but when you think about it, we've replace one simple call to save, with two calls to dettach then reattach. Where's the savings? Plus, calling save is more more natural.

So if we go back to open/closing session at DAO, lazy loaded references throw exceptions because they're no attached session. Which orignally lead me down the path to using FlushingSpringSessionInViewFilter. I'm inclined to solve this with an AOP interceptor for each method that requres lazy loading, right now only one, I may just hack it until I can get my head around Spring's AOP.

Resetting database on sipxchange 2.4

2005-02-25T05:23:00.000-08:00

I had to find out how to reset db on 2.4 system, here's how:
su - postgres
dropdb PDS
cd /home/sipxchange/configserver/etc/db/pgs
./p_xchange.sh