httperf status - 2008-12-30

I have to acknowledge that I have been dropping the ball in regards to completing the next release of httperf. While there has been a good amount of work done on this tool since the last release (0.9.0) I just haven't gotten around to wrapping it up yet.

Anways, I had better get to it now.

Check CVS for the todo list.

Other than simple procrastination, one major reason I put off development was that autoconf support for detecting the c99 constructs I was using wasn't integrated until 2.62.

Then I was going off on tangents, trying to implement my own libevent based http client library (which is definitely a useful project in my own opinion) and working to get mtp support introduced into the kernel.

Anyways, where we stand today is that the head of cvs is not terribly stable at very very high request rates. Maybe this is because httperf has never been able to achieve the rates that we are now reaching before and it has exposed some hithertofore unkown issue.

Whatever, I'll track it down.

As for the libevent migration. Well, Adrian Chadd was kind enough to do a partial port. However there is much more work to be done in this area such as migrating timers, timeouts, dns and http control into the libevent core loop.

Long term, httperf will dramatatically shrink in size. Event logic, the http client, and metrics collection can all be loaded onto libevent and some (as of yet unwritten) plugins. httperf will be a set tools implementing various workload generators and may further evolve into a workload language implementation.

A spin with gnome-blog

I was poking around the software repository for opensuse 11.1, and came across a tool called gnome-blog. This is a panel applet which is supposed to allow me to post to my blog directly from the desktop rather than the supposedly tedious process of opening the blog website and posting from there.

Anyways, the tool is supposed to allow me to drag and drop images inline with the text, so I figure I will give it a whirl.

This version of gnome-blog is from 2006, and appears to be the most recent, so I suppose that development has stalled. Google has failed to deliver anything of interest more recent than that.

Update: I am not impressed. So long gnome-blog.

-Ted

Linux Kernel Space Media Transfer Protocol

So after a couple years of pondering I have finally begun work on implementing a kernel space driver for the media transfer protocol.

Yes, yes, this is my first foray since university into kernel space, but I have had enough of the user-space solutions to implementing the media transfer protocol. Hundreds of devices have now implemented the darn thing (including my iriver clix if you recall), and what's more the USB.org folks have finally published the 1.0 version of the standard.

First here is a bit of the technical reasoning.

1. Already two distinct user-space implementations which both share similar drawbacks. libmtp, and libgphoto


2. Dependant on libusb which has stalled in development


3. Race condition in device connection, first application to detect and connect blocks any other software from using the device until connection is dropped

Etc, the current implementation suffers from all the problems that user space drivers suffer from in the monolithic world (even the microsoft implementation has this going on). Also, it's about time I get my hands dirty in the kernel for pete's sake.

So how is this thing going to work you might ask?

Well, here is how I see it right now, and others are welcome to chime in on this.

The protocol is implemented partly in kernel space, and partly in user-space depending on necessity and appropriateness.

Kernel Space Driver (mtp):
Connects and manages all operations:
Exposes file hierarchies as a mountable file system
Implements protocol for transferring straight files.
Issues device handles to userspace which can be used asynchronously (no more device pointers!)
Automatic device connection

User Space Library (libmtp-media)
Implements protocol for managing media (including meta-data) (music, pictures, video, etc)

User Space Library (libmtp-cal)
Implements protocol for managing contact and calendar (vcard, ical, etc)

Essentially the protocol implementation will be separated across a variety of locations.

Anywho, after an hour or so of fiddling, here is the latest line from dmesg:

usb 4-5: new high speed USB device using ehci_hcd and address 9
usb 4-5: configuration #1 chosen from 1 choice
mtp: USB Skeleton device now attached to USBSkel-192
usb 4-5: New USB device found, idVendor=4102, idProduct=112a
usb 4-5: New USB device strings: Mfr=1, Product=2, SerialNumber=3
usb 4-5: Product: iriver clix
usb 4-5: Manufacturer: iriver Limited
usb 4-5: SerialNumber: 41dff81c000000f33230303530313031
usb 4-5: USB disconnect, address 9
mtp: USB Skeleton #192 now disconnected

Not much, but hey it's the beginning of the road here.

Lastly, if anyone wants to employ me to design this, I am certainly on the market.

-Ted Bullock

How to synchronize a Nokia 6301 in Linux - Investigation

This afternoon I received my brand new Nokia 6301 cell phone. In many respects, I really do like the thing.

This nokia smartphone has a wifi antenna which allows it to be used as a VOIP handset if you are in range of an appropriate network and also supports all the normal functionality of your typical cell phone. It also supports Bluetooth and has a mini-usb adapter.

When physically connected to a modern linux machine (in my case openSUSE 11.0), it initially identifies itself as a USB Mass Storage device and the desktop responds appropriately by mounting it like any other USB drive. However, the phone specifies a couple of other connectivity options, including "Data Storage" (USB Mass Storage), "Media & Printing" (MTP) and "PC Suite".

When you choose "Media & Printing", the device switches to MTP mode (Woohoo, go libmtp!) which libmtp supports very nicely, although after perusing the datastructure, I am not clear which folders are the "correct" folders for which type of file. I guess this would just require some testing inside of Windows to see where the nokia tools put things. That said, this also means that f-spot should work pretty much out of the box.

Anyways, the third option "PC Suite" puts the device into a state which the nokia PC Suite application can use to synchronize the phone with Microsoft Outlook or Lotus Notes. This third area is the most interesting for me at the moment. I want my Evolution Contact List and Calendar to synchronize with with the phone so that I don't need to pull out my laptop just to check if my flight to New York leaves at 10:45 or 11:45.

There do seem to be a number of tools for solving this problem, however they are not particularly explanatory, nor have I actually been able to sync my calendar, todo list or contact list to the phone in yet in Linux.

So, that pretty much summarizes my experience to date with the Nokia 6301 in Linux.

I definitely would appreciate someone pointing me to documentation on how to get my calendar out of the PC and onto the phone.

The problem with current design software

The more experience that I have with how engineering and electrical design is done, more I am astonished at how much effort goes into problems that are fundamentally trivial.

For instance, when designing a schematic, I need to spend a considerable amount of time figuring out which terminal block contacts are available, and whether or not the final wiring layout is electrically consistent with the schematic.

I don't care about that!

No kidding.

I cannot believe that I need to do that in this day and age. I should be able to plug device one into device two and my design software should just know how to make it happen. Period.

At the end of the day I want to be able to say to my machine

"Connect these devices in this order"

I should be able to just figure out basic system layout, and all the wiring, schematics and general fiddling should be just figured out for me.

openSUSE 11.0 and the nouveau driver

I recently got my nVidia dual-head Quadro NVS 280 PCIe graphics card functioning in openSUSE 11.0 with the open source nouveau driver!

Originally I was inclined to use the proprietary nvidia driver since the nv driver does not support multi-head setups with my particular card. However, the legacy nvidia driver is not functioning at the moment with the current kernel iteration of openSUSE 11.0. So after a few hours of frustration, I blew away the proprietary driver and began work on installing nouveau.

The binary version that is comes packaged for the distribution won't let me start X for some reason or another, and besides, I wanted to try out the latest and greatest developments towards the driver.

Following the installation instructions was straightforward after I installed the necessary devel packages that come with the distribution. And within a few minutes I had a functioning desktop cloned across my two monitors.

The next step was enabling telling randr to spread the desktop across both monitors, by folling the randr instructions on the nouveau wiki.

Clearing out a good bit of the autogenerated cruft from the xorg.conf file provided me finally with a functioning extended desktop. All this with the open source driver, and a bit of perseverance.



Thanks to the folks on IRC! Very helpful, and best wishes!

-Ted

New version of httperf coming soon (httperf 0.9.1)!

In spite of the fact that I barely spend any time these days working on httperf, I am pleased to note that activity has not been totally absent.

Pretty quickly we'll be releasing a new version of httperf with a number of enhancements.

The most dramatic change came from Adrian Chad, who implemented a basic port to the libevent notification system. This has dramatically improved the performance of the tool (like in the realm of "orders of magnitude").

New period variation option --period=vT1,D1,T2,D2....Tn,Dn

The idleconn tool is now optional (disabled by default), and can be built using the --enable-idleconn configure option

New options to print result data in new formats (CSV and Key=Value), however this is still being tested.

The idleconn program has been re-written to use libevents non-blocking dns and connection API.

Also, a number of memory leaks have been plugged with a re-write of the timer system.

libevent is now a build requirement.

httperf will try to compile itself with the system compilers c99 option if available.