httperf status - 2008-12-30

I have to acknowledge that I have been dropping the ball in regards to completing the next release of httperf. While there has been a good amount of work done on this tool since the last release (0.9.0) I just haven't gotten around to wrapping it up yet.

Anways, I had better get to it now.

Check CVS for the todo list.

Other than simple procrastination, one major reason I put off development was that autoconf support for detecting the c99 constructs I was using wasn't integrated until 2.62.

Then I was going off on tangents, trying to implement my own libevent based http client library (which is definitely a useful project in my own opinion) and working to get mtp support introduced into the kernel.

Anyways, where we stand today is that the head of cvs is not terribly stable at very very high request rates. Maybe this is because httperf has never been able to achieve the rates that we are now reaching before and it has exposed some hithertofore unkown issue.

Whatever, I'll track it down.

As for the libevent migration. Well, Adrian Chadd was kind enough to do a partial port. However there is much more work to be done in this area such as migrating timers, timeouts, dns and http control into the libevent core loop.

Long term, httperf will dramatatically shrink in size. Event logic, the http client, and metrics collection can all be loaded onto libevent and some (as of yet unwritten) plugins. httperf will be a set tools implementing various workload generators and may further evolve into a workload language implementation.

New version of httperf coming soon (httperf 0.9.1)!

In spite of the fact that I barely spend any time these days working on httperf, I am pleased to note that activity has not been totally absent.

Pretty quickly we'll be releasing a new version of httperf with a number of enhancements.

The most dramatic change came from Adrian Chad, who implemented a basic port to the libevent notification system. This has dramatically improved the performance of the tool (like in the realm of "orders of magnitude").

New period variation option --period=vT1,D1,T2,D2....Tn,Dn

The idleconn tool is now optional (disabled by default), and can be built using the --enable-idleconn configure option

New options to print result data in new formats (CSV and Key=Value), however this is still being tested.

The idleconn program has been re-written to use libevents non-blocking dns and connection API.

Also, a number of memory leaks have been plugged with a re-write of the timer system.

libevent is now a build requirement.

httperf will try to compile itself with the system compilers c99 option if available.

httperf and libevent and the future of everything

Firstly, congratulations to Adrian Chadd for starting the httperf port to libevent. Already we have seen some massive performance enhancements. CPU usage on my opensuse 11.0 workstation has improved about %3000 percent.

Certainly there is some stability work to be done. I am currently getting segmentation faults with very high request loads.

The current implementation does not use a persistent event loop, which means that the other useful features that libevent provides are not available (timers for example).

Something I want to see developed going forward is an asynchronous http client library around. The http work already done with libevent is certainly a start to this, but is far from complete. Essentially, what I want is a client api where I can perform some http client action asynchronously without needing to deal with any messy details.

The intention here is to foster a deal of code-reuse between http implementation projects that all use libevent at the core. After that, portability sits firmly on libevent's shoulders (where it belongs).