Blog

Evaluating Add-Ons - What is the risk of adding a given dependency?

ross — 2008-12-01T19:52:46Z

As often happens, a client asked me about adding a given package to their buildout. I just realized that I have a standard response to this so that might be something worth documenting for the wider community of admins and integrators. My criteria are as follows roughly in order of priority:

Good backing:

Find the author or maintainer on the plone.org/products page or on PyPI. Are they a company or individual that is widely known in the community?

Recent versions supported:

Does the add-on support recent versions of its own dependencies? For a Plone package today this means checking if it support Plone 3.

Final release:

Does the package have a final release that supports recent versions of its dependencies?

Released as an egg on a package index:

Is the add-on packaged as an egg and available on an easy_install accessible index such as PyPI?

Test coverage should, of course, be criteria #1. Sadly, we don't have a good enough test isolation story with ZTC and PTC, test failures in the client's buildout don't necessarily mean the add-on is broken. I know there's been talk about integrating test coverage data into plone.org/products but it's not here yet. The holy grail would be some sort of integration with distutils/setuptools. It would be great if I could do "python setup.py test --coverage" so that a subsequent "python setup.py register sdist upload" would incorporate the coverage data into the release meta-data such that it would be reported on the PyPI page. Certainly any such solution could be circumvented by the releaser, but I think that would be very rare, that it would more often encourage more maintainers to have good test coverage and this would be a huge win overall.

It's also interesting to note that I find recent version support more indicative of risk than final release status. This comes from experience. I've worked on too many projects of all sizes that had production sites running release candidates, betas, alphas, and even some development versions. One conclusion would be that this means that the software is of poor quality. I don't think that's it. I think it's that we, as a software ecosystem, are bad at releasing. Now eggs are certainly helping with this, but I think we also need to start an aggressive honor/shame (carrot/stick) campaign about releasing. Actually, maybe we should have an honor/shame campaign about add-ons in general. Perhaps a montly post to planet plone and the mailing list with a Hall of Fame and a Hall of Shame? :) Something that considers release status (how long has it been since that beta release? 3 months? Maybe you can release a final version?), test coverage, and meta-data cleanliness, etc.

Balancing DRY and Readability - Some complexity belongs in your editor/IDE

ross — 2008-11-25T15:13:19Z

I was writing a combination of setuphandlers and functional test fixtures recently when I found myself doing something noteworthy. I was converting both apparatus from a set of loops over content type names and prefixes to a few utility functions with lots of repetitious calls. Why was I violating DRY? Because I couldn't make sense of things! Oh yeah, DRY should never compromise readability. :)

It's definitely possible to take DRY too far leading to such slap-your-forehead moments as this. I think that setup code, whether for install or a test fixture, is particularly prone to this. I've both written a lot of setup code and read a lot of it in the wild where a bunch of complex structure is introduced in the name of DRY.

What I ended up doing was writing a small number of utility functions whose call signatures were intuitive and readable. Then I used a bunch of complex "M-x query-replace-regexp" commands in Emacs to convert the complicated structure into a simple, readable, flat, but nonetheless very repetitious function calls. Later when I needed additional set up, I used the same complex editor command to accomplish the task.

It occurred to me that what I was doing was taking unreadable complexity out of code where I, let alone someone else, couldn't remember or read what was going on, and i was moving it into my tools. My tools I use every day so I remember how to manage the complexity and I'm not condemning others to learn the complexity when I put it there. I think this makes a simple guideline for balancing DRY with readability. Some complexity belongs in your tool usage, not in the code.

Community Introspection - Same risks and rewards as for the individual

ross — 2008-11-22T03:15:54Z

One of the things I love about being a member of the Plone community, is our capacity for introspection. Like all things, however, it has its place and its limits. I've been contemplating how we might find those limits.

At the recent World Plone Day, I had the opportunity to hear Donna Snow's story in more detail. She was one of the "old school" skinners who was hurt by the Plone 3 changes. She has been assimilated. :) She's now a stronger advocate than ever. That's a great example of how the Plone communities' introspection is an asset. We didn't flinch, we incorporated.

OTOH, sometimes I think our community could have a bit more patience with the significant though perhaps unsexy transitions that the Plone stack is in the middle of. We're no longer the new kid on the block and so we lose some of our sex appeal. We're now a successful project whose greatest challenge now may well be to gracefully manage the history of that success while we continue to improve.

It may not be so scintillating to participate in keeping a successful project nimble as it was to participate in the rise of a shining new success. I know, however, that I at least will be very pleased with my time if I'm now a part of the subtler success of standing up well to new challenges and challengers.

I very much value our community's capacity to make constructive use of criticism and to be critical of ourselves. I just think it would help morale to also raise enthusiasm while we're doing so.

Nonprofit Software Development Summit - 3 brilliant days with cracked out LAMP folks

ross — 2008-11-21T03:45:34Z

I got back from the 2008 Nonprofit Software Development Summit in Oakland, CA last night. I had a total blast. It was simultaneously intense and relaxed which is an impressive achievement. It was also very surprising for someone who doesn't expect a tech conference to be intense. I generally pride myself on endless energy. The last day of the conference I didn't even have the energy for the final round of libations. I drove straight home and collapsed. :)

I don't want to offend other conferences or formats, but I hadn't realized how much I don't enjoy the traditional conference format. Now that I've experienced such an event where I was genuinely and organically engaged for 80-90% of the time, I recognize that I have always been very bored at other conferences. Nate tells me that bar camps or other events with a tendency to use the term "camp" have a similar format. Needless to say, I'll be keeping an eye out for such events.

This event didn't allow laptops at any of the sessions or main gatherings, and I didn't even miss it. I'd be dead of boredom at a traditional conference if my laptop were forbidden. I certainly didn't miss PowerPoint, which was also forbidden. This was also the first conference I presented at. It occurs to me how miserable it would have been to present to a room of downward facing eyes and tippity-tappity keyboards. When I thought about it further, I realized it's similarly miserable to sit in a session dealing with the conflict between boredom and the genuine desire to be engaged for both the presenter's sake and mine. The way this event handled this conflict was to limit presenters to 5-10 minutes of intro time after which it was all group discussion. It was very enjoyable and engaging both as an attendee and as a presenter. I assume all this is obvious to those who already enjoy this format. :)

I gave two presentation. The first was a contemplative session on Risks and Opportunities of Cloud Computing. I wanted to see how much traction my thoughts would get when it comes to adopting CC or PAAS while also building in structure to protect code, data, and vulnerable clients such as non-profit organizations. I found that most are still thinking about the question in terms of either "get over it and get on board" or "you'll get my app into the cloud when you pry it out of my cold, dead hands" or "you'll never find my app or data stuffed in my mattress". I want both protection from the risks and I want to take advantage of the opportunities. My hope is that open source frameworks that provide abstractions of core cloud services along with additional libraries offering solutions to common needs may be able to realize cloud apps that can be feasibly ported to other providers. These frameworks could then be a fulcrum for leverage to keep providers honest. This idea gained next to no traction with my attendees who wanted to go in other directions. This, in and of itself, is informative. If I'm wrong about using frameworks, or if I'm right and few are ready to move in that direction, either way there's a lot of work ahead.

For my second presentation I thought I'd play to my strengths: Best Practices for Plone Performance. Of course, I proposed this talk having never attended this conference before. The attendees of this conference were, I'd guess, 90% LAMP people which means the CMS world was dominated by Drupal. Joomla was a distant runner up. There were also a fair number of Ruby, and Rails, people there. The only people I met using Python, Zope, or Plone were Nate Aune and the good people from The Open Planning Project. As such, this talk wasn't overly attended. :) Just the same, a potential new Plone adopter was there and it was interesting to compare scaling wisdom between camps. From what I could glean, Plone has a much better scaling story in many senses. It's also worthy of note that nearly every time I mentioned Plone, the person I was talking to would mention ONE/Northwest. They have impressive recognition in the community.

Speed Geeking is a great for demonstrating technology and evangelizing or advocating all at once. This format allows a presenter to convey their enthusiasm for a project where the audience is consumers while eliminating much of the boredom that typically occurs when an audience is compelled to listen to one person talk. I found myself wishing that I could cover the very rich Plone feature set in such a limited 3-4 minute format. If I've noted one thing about what impresses people about Plone, it's how much they get OOTB. If you can get someone to pay attention long enough to cover the feature set, they're almost always surprised. I can't count the times I've heard, "I had no idea Plone could do so much." Unfortunately, you can't demo all these features in 3-4 minutes. I was thinking it would be good to have a slide show amongst the Plone marketing materials that could cover "What Plone can do" in 3-4 minutes. Anyone interested in working on that with me?

The LAMP people at the conference were all very open and there was a very pleasant absence of evangelism. I even noted that as the conference went on, everyone seemed to make a point of adding Plone to their list of token CMSes when speaking as they became more aware of our presence. All told, I think it was very productive for all sides and so much of that is due to the community of the event. Everyone there had gobs of energy, was genuinely open and friendly, and everyone seemed to be passionately committed to exciting and productive projects. I started as an attendee, came to see myself as a guest, and left feeling as a member. Quite an achievement indeed.

I'll blog more later about the sessions I attended.

Running Tests in Egg Buildouts - A quick note about a gotcha with plone.recipe.zope2instance.ctl

ross — 2008-11-15T04:25:58Z

Recently I made an egg of mine into a buildout for self-contained testing and I ran into a gotcha that took me too much time to figure out, so I thought I'd document it for others.

The plone.recipe.zope2instance.ctl test runner excludes the buildout root from the test paths. It gives the reason in the comments:

# XXX Somehow the buildout root gets on the sys.path. This causes
# weird problems, as packages like src.plone.portlets.plone.portlets
# are found by the testrunner

Since a ZopeSkel generated egg doesn't create a "src" directory to contain the actuall python packages, the egg root would need to be on the test path. When the egg is made into a buildout then all these factors collide to make the egg I want to test invisible to the test runner.

I worked around this by moving the actual python packages to a "src" directory. I always thought this was more proper anyways. Maybe ZopeSkel should be changed. At any rate, here's the setup.py additions that were necessary:

packages=find_packages('src', exclude=['ez_setup']),
package_dir = {'':'src'},

Hope this saves some trouble!

My Borrowed World Plone Day - How I crashed Donna Snow's presentation

ross — 2008-11-07T22:05:55Z

I'm spending my World Plone Day at Donna Snow's presentations at Sunnyvale, CA. It's been a blast.

Donna started out describing her participation in the Plone community and her company, C Squared Enterprises (C2E). Then she moved onto a slideshow with a good overview of what Plone is and some of the best reasons for adopting it. Next she gave a live demonstration of a Plone 3.1.6 site and managed to convey a lot of the very feature rich set that Plone brings out of the box keeping everyone engaged the whole while. An excellent presentation.

Donna graciously invited me to make some contributions to the presentation from a developer's perspective. It gave me the chance to talk about some of the technical underpinnings of the Plone stack I most admire. The internationalization and translation capabilities seemed to get a lot of favor, big surprise there. :) I listed WSGI and Deliverance theming as some of the more likely future directions for Plone and was surprised how much interest that got. I definitely enjoyed participating.

Here are some of the questions asked:

What other CMS are there?
What is Plone's market share or adoption rate? Is it growing or shrinking? How does it compare with other CMS?
How is translation done?
Are live backups possible? Can they be done TTW?
Can we see the developers view of a template?
I heard Plone is slow?
What's this Zope 2 and Zope 3 stuff?
Can you make the calendar portlet start the week with Sunday?
What is the granularity of the security framework?
How can I integrate external database queries with Plone?

It was a first for me sitting in a room fielding questions from people who were doing their initial evaluation of Plone. While none of the questions were surprising, it was nonetheless very informative.

It was very generous of Donna and C2E to pay for the space and the materials and to donate all the time. I think it was worth it. Best quote of the day?

I'm a convert.

It looks like the second session is turning out to be more of an installfest. :)

Cloud Busting - What does it mean to be a good cloud citizen?

ross — 2008-10-23T05:43:52Z

I know I'm not alone, but I've been thinking a lot recently about what the cloud means for rich web applications like CMS's such as Plone.

Firstly, I find Google App Engine (GAE) much more inspiring than Amazon's Elastic Compute Cloud (EC2) when it comes to thinking about cloud computing. Kinda funny considering that EC2 actually has "cloud" in its name while GAE doesn't.

It seems to me that EC2 isn't really a cloud. For my money, EC2 is just a really big virtual hosting provider that has done a great job of limiting hosting costs by bringing efficiency to client relationships and expectations. They do this by simultaneously limiting their responsibility and clearly defining their remaining responsibility in accessible APIs. They exploit a very interesting and relatively new technology sweet spot to do this, virtualization. This has the potential to become the *foundation* for cloud computing,but is not cloud computing in itself. The strategy seems to be to provide the foundation, open up the technology race, and wait and see who succeeds in building good clouds on that foundation. I kinda like this strategy, but I do see some problems with it. Of course, mostly,I just don't like thinking about hardware, virtualized or no. :)

Firstly, computing in the cloud is definitely going to require changes from the code that operates in the cloud. EC2 enables application builders to construct large distributed web applications without ever making their code into good cloud citizens. That my not be a good thing. Similarly, various parties are building true cloud frameworks on top of EC2. It may be that an inferior cloud framework places fewer requirements on the hosted code, thus gaining more adopters,thus starving better cloud frameworks.

GAE seems to come at the cloud from the other direction. Rather than providing all options on something that isn't yet a cloud as EC2 does,GAE provides a true cloud but with very limited options. GAE too exploits a technology sweet spot, but a very mature and well established one, Python. The strategy there would seem to be to open up the technology race in the other direction inviting framework/application developers and GAE to come together over time. That is, framework/application developers will make their code better cloud citizens guided by the restrictions of a true cloud and GAE will become richer and less restrictive as Google sees where to put its resources in terms of extensions or flexibility. I've heard GAE called a platform for toy applications. It seems to me it's not so much a toy platform as a very serious platform in its adolescence.

Now, what about large, feature rich applications like CMS's such as Plone? The ideal large, feature rich application still has a lot of code, the code that implements and integrates all those features. As such, I'm going to set aside the question of whether or not a given large, feature rich application is *too* large, just for the moment. Such waste definitely becomes even more painful in the cloud and good cloud citizens should definitely continually evaluate their code waste even more vigilantly, it's just not what I want to consider right now.

I think specialization is going to be one of the most important aspects of cloud computing. In order for a cloud to be flexible and transparent while simultaneously achieving responsiveness and efficiency, applications are going to have to cooperate with the cloud so that requests can be handled by portions of the cloud that are already more or less familiar with handling those requests. Good cloud citizens will have to factor themselves such a way as to support this. Once code running on the cloud has factored itself into specializable requests, the cloud can be left to do the specialization as they see fit. The cloud provider is the party most interested to squeeze the most possible efficiency out of the cloud so the implementation of specialization should really be left to them. Of course, once again, mostly I just don't want to think about specialization, too close to the dreaded hardware. :)

AJAX to the rescue! It seems like AJAX first burst onto the scene to improve client side performance. If you replaced a portion of the DOM with the results of an XMLHttpRequest, you could avoid the page reload time cost incurred by the *browser* and the cost of the user experience disjoint incurred when their browser renders a new page. Using AJAX to compose a page from multiple requests is, however,potentially *more* valuable to specialize things on the *server* side. The win here is not so much the AJAX on the client side as is the capacity to break a page up into multiple specializable requests for the server side.

Specialization becomes most important with large, feature rich applications like CMS's. Without specialization, the application eventually requires too much in the way of resources to handle any given request and becomes a very bad cloud citizen.

Effective specialization also requires that applications be frugal about how much of their code needs to be loaded to serve a given request. This cuts down on memory consumption but perhaps more importantly, it cuts down on startup time when an portion of the cloud that is not yet specialized is called upon to serve a new request. It seems like any large application worth it's salt uses some sort of modular registry (plugins, components, whatever). I think it's vital that any such registry that hopes to be a good cloud citizen shouldn't load the code behind any given lookup until the lookup first occurs.

So lets talk about Plone. At the moment, Plone has a lot of work to do before it becomes a good cloud citizen, but it seems like it will be possible to get there. I will be blogging about this in more detail in the near future, but here's a sketch.

I wrote a small test that ran against an *existing* Plone trunk site and simply loaded the front page in a testbrowser. I confirmed that zope.testing coverage reporting does not report on modules that aren't imported as a part of the test run. Then I ran that test with a coverage report to get a sense of how much of the code that is imported for Plone startup is actually used to serve that page. Finally, I removed the coverage numbers for all Python standard modules and any third party dependencies in order to get a sense of the import waste in the Zope/CMF/Plone stack:

lines	cov%	lines uncovered	lines covered
167,752	40.00%	100,643.21	67,108.79

Firstly, 67k lines of code to serve up the front page seems like too much, but we're aware of that. The limbo between the more monolithic Zope 2 and the much more elegant successor Zope 3 is probably a significant part of that. I'm going to do some more detailed examination of the coverage report, but here is the CSV of the simple coverage numbers for those interested.

The thing I find most interesting is how much is imported code isn't used. I've been thinking about this for a while. While some of this is certainly import waste from unused portions of the more monolithic Zope 2 and Archetypes stacks, I think much of it may also come from ZCML and the Zope Component Architecture.

The ZCA gives us a wonderful registry for a very modular architecture but using the ZCA requires importing all the code for everything registered. Some significant advantage might be had by using a ZCA registry that didn't import the code until the first time that given lookup is performed.

There is an interesting exception, local site managers. Since local site managers store their registry in pickles, it may be that the code behind the component isn't loaded until the lookup unpickles the registration. I haven't confirmed this, but even if this isn't the case and the whole registry is unpickled at once, it might not be too hard to make each registration its own pickle. Marius mentioned an effort to reduce ZCML load time by pickling the configuration actions. Maybe it's possible to direct the ZCML actions at a local site manager instead which could then be persisted and used as the global site manager. Then loading the ZCML at startup would be unnecessary. In essence, this would mean *compiling* ZCML into a site manager pickle before deploying the applictaion to the cloud. This notion of compiling configuration for deployment brushing shoulders with pickles has inspired some other interesting notions I hope to blog more about soon.

As for AJAX, Viewlets seem like an ideal framework to exploit for AJAX composed request specialization. I might be feasible to replace viewlet renderers with ones that insert JavaScript into the containing page which will load the viewlets in separate, specializable requests. Someone's already tried it.

As for Plone on GAE, there are other, possibly larger hurdles, not the least of which is Zope's use of C extensions. I know work has already been done to lessen dependency on Acquisition wrappers, but I don't know how close that brings us to eliminating the C extensions there. As for persistent, I don't know if what is done in persistent.cPersistence can be done in python but I have heard noises that the effort to port ZODB to Jython may yield fruit here. It also seems like it might be possible to use some of the ORM libraries out there to implement a python only version of BTrees that uses tables on the backend. At any rate, all of this really is pie in the sky at this point, but I have a 10% Pie in the Sky Manifesto and an awful lot of good things have started with such thinking. I'll be blogging on this more later.

At this point I'm more interested in what computing in the cloud means in the more general senses detailed above but I'd love to hear any thoughts on any of this.

Fine-Grained PM on Small Projects - Increasing control while reducing project management costs

ross — 2008-10-03T18:58:14Z

I made a mistake recently that I've made a few times in the past. Time to change something. :)

A small project I was working on had transitioned to a phase where very little of my time was required. Unfortunately, there was still a whole lot of email correspondence flying around. I wanted to avoid putting much time into PM when it would be disproportionate to the amount of development time. Several of the emails contained explicit direction some of which superseded others. At one point, I got it wrong in my head and did some work out of priority order. In this case the situation was easily remedied but I'd really like to have a more robust approach to this problem before I make the same mistake when it's not so easily remedied.

For large projects an issue tracker is a great PM tool. A priority field can be used to order tickets. Since so much time is being put into so many tickets, it tends not to be valuable to have explicit ordering to the degree that tickets must be worked on in a completely linear order. IOW, if ticket foo and ticket bar have the same priority, it tends not to be worthwhile to worry about which one to work on first given that the time for each is small next to the amount of time being put in on all tickets. Every team member can just bookmark the link to the canonical report that tells them what tickets to work on in what order and PM time can be minimized.

For small projects, however, things can be very different. It is very important to have explicit ticket ordering, but more than that it is often very important to track other explicit and *arbitrary* direction on a per-team-member basis. As such, an issue tracker isn't really a good match, even if it supported arbitrary ordering. Email is both noisy and lossy unless an inefficient amount of time is put into tracking it.

I was brainstorming on this with a favorite project manager, Sally Kleinfeldt, when she hit upon the embarrassingly simply solution. Use a canonical shared document somewhere for each team member requiring direction that is narrative rather than structured in nature. IOW, I get my own document that the client, the PM, and myself can edit which will always represent all of my current direction. If there are other team members requiring direction, they should each get their own page.

This document can link to particular tickets in the issue tracker to affect arbitrary explicit ordering along side any of the kind of arbitrary narrative direction that tends to be a part of small projects. Email can still be used to track when my direction changes, but never again will I have to spend a half hour digging through emails to double verify that my next and only task is to put 15 minutes into a small fix. I simply go to my page, and move on. This document could be a google doc, a wiki page, a special ticket in the issue tracker, whatever, so long as it is canonical.

It's very simple, and so probably has been in use before, but I'm nonetheless very excited about it as a way to improve efficiency on small projects.

Testbrowser Redirects with Fragments - Using zope.testbrowser when a redirect URL includes a fragment

ross — 2008-08-09T00:29:35Z

I was trying to add functional test coverage for commenting on content when I kept getting 404 NotFound errors. I tried it in a real browser and everything worked just fine. After a little googling, I found an answer in a post on the zope3-dev mailing list:

Note that the fragment identifier is *never* going to be passed to the server by a "real" browser: instead, the browser strips of the fragment part, submits the remainder of the URL to the server, and then does a search for the appropriate '<a name="bar">' element *on the client side*.

I'm not sure how this is pertinent to the problem you have found, but Zope will never traverse '#bar' or '%23bar' in the real world.

It seems like maybe testbrowser should not send fragments to the underlying mechanize browser. But failing that, for now I just add a note to my tests about why we expect a 404, NotFound error in our tests that we won't see in a real browser.

Hope this saves someone else the search time. :)

Re-using SSH Connections - Automatic master SSH connections for sharing and multiplexing

ross — 2008-07-26T07:07:22Z

Yet again I've discovered that somethign I've been dreaming of forever already existed for just about that long. :)

I use SSH for so many things these days that the startup time for connections really begins to add up. Enter OpenSSH connection sharing. See that blog post for details. Also see this thread on the SVN development list for details of a workaround for an svn+ssh issue.

I was still left wanting, however. The lion's share of the time I want to save re-using SSH connections is for svn+ssh. The SVN workaround requires that I think enough ahead to start the master for every host I connect to. I connect to a lot of hosts and they're constantly changing so I wanted something I didn't have to think about. Also mildly annoying is the fact that the first SSH session to a host would become the master and couldn't be closed or killed without killing all slaves and, of course, killing the master.

I wrote a wrapper for SSH that would start the master in the background if need be and then run the ssh command as a normal slave. This solution allows the individual connection to close or be killed without affecting the master. It also obviates the need for the svn+ssh workaround. Enjoy:

#!/bin/sh
## Wrapper script for ssh that backgrounds the control master if one
## isn't already started.

## the real ssh
SSH="/usr/bin/ssh"

## optstring assembled from `man ssh`
optstring="+1246AaCfgKkMNnqsTtVvXxYb:c:D:e:F:i:L:l:m:O:o:p:R:S:w:"
## get ssh options
opts=`getopt -- "$optstring" "$@"`

## the non-option args follow "--"
## convert to an array of IFS separated strings
args=(${opts#*--})

## the host is the first non-option arg
## use eval to process the quoted strings returned by getopt
host=`eval echo ${args[0]}`

## if the master isn't running, start it in the background
$SSH -q -O check $host 2>/dev/null || $SSH -MNf $host

## replace ourselves with the reall ssh call
exec $SSH "$@"

Spaces in User Names and psvn.el - Handling SVN user names with spaces in emacs' svn-status.

ross — 2008-07-25T17:38:22Z

I recently started on a project using svnserve's --tunnel-user option where many of the committers have user names with spaces. Much to my chagrin, my beloved svn-status in emacs choked on those user names un able to tell where the user name ended and the filename began.

Then I found a comment in the current version of psvn.el detailing how to work around this. Just be sure you have the current version on your load-path and add something like the following to your ~/.emacs:

(setq svn-user-names-including-blanks '(\"feng shui\" \"mister blank\"))
(add-hook 'svn-pre-parse-status-hook 'svn-status-parse-fixup-user-names-including-blanks)

Strange Critters - A few strange critters I ran into hiking

admin — 2008-06-30T03:29:37Z

Starting with the disclaimers, I'm not a photographer nor much of a naturalist. As a story teller, I love photography and hate snapshots. My hypocrisy justifies the photos below as "documentation" which also serves as my excuse for their quality or lack there of. I'm an avid hiker because I love covering terrain. I do love to look at all the flora and fauna as I hike, I just know very little about it. So I'm making no claims that anything is special.

Over the last few months of hiking I've run into a few strange critters and snapped some pretty crappy photos of them with my phone. Anyone have any ideas what these are?

Here's an oversize centipede-like bug found on Ridge Road in Big Basin. It's hard to see, you might want to open the full sized image and zoom in.

I also ran across a couple of small brown snakes or worms. They were dead when I found them. I don't know if these are the same critter or not.

This spider was so remarkable. Unfortunately, the picture came out blurry. Sorry!

Foxglove - White foxglove from East Ridge Road in Big Basin.

admin — 2008-06-30T03:03:46Z

At any rate, I noticed a flower I hadn't taken note of before and it kinda stuck in my head. Later, I ran into Scott Peden and asked him about it since he definitely does know this kind of stuff. He guessed foxglove from my description and was, of course, right. The next time I passed them, I snapped a few pictures with my phone.

Here's another white one:

Here's a bright purple one:

And finally a more lavender one I ran across a little later:

Eagle Rock View - Faux panoramic from Eagle Rock in Little Basin

admin — 2008-06-30T03:58:52Z

Try zooming in on any of them, they actually came out alright.

Adding testbrowser Form Inputs - Mock dynamic form inputs with zope.testbrowser

ross — 2008-06-01T18:48:20Z

When writing functional tests that cover forms with elements that are dynamically manipulated from JavaScript, such as the ReferenceBrowserWidget, I often simply didn't cover those elements becuase it simply wasn't worth it to add Selenium or zc.testbrowser.real as a testing dependency for what are usually very small pieces of what is being tested.

What I've always wanted was a way to reproduce the effects of the dynamic manipulation in the testbrowser. When I was recently bitten by a bug that I didn't catch for lack of functional coverage of a ReferenceField, I dug into the ClientForm package that zope.testbrowser uses and found the HTMLForm.new_control() method. This is the method that ClientForm uses to construct the controls that zope.testbrowser provides lookup for.

Since Firefox allows you to inspect the DOM either by selecting the elements and using "View selection source" or with Web Developer's DOM inspector, you can ferret out what the dynamic manipulation created and use that information to construct the HTMLForm.new_control() call that will reproduce that. The following example would add the foo object to the related items on another object's edit form.

>>> form = browser.getForm(name="edit_form")
>>> form.mech_form.new_control(
...     type='checkbox',
...     name='relatedItems:list',
...     attrs=dict(checked='checked', value=foo.UID()))

See the ClientForm source for more details.

BTW, it seems the the ReferenceBrowserWidget includes an empty string on submission of the form. This empty string case seems to be handled in the field's set() method and not in the widget where it seems like it should be. Not knowing this, I implemented a custom mutator with a bug that I didn't catch because I didn't have test coverage on the form submission. Now I do. :)