hudson won’t work with tomcat security manager enabled

I tried installing Hudson on by copying the war to the webapps folder on Ubuntu (what a mess it makes of a tomcat install.  You should NOT, repeat NOT use apt-get to install Tomcat, among many other things.)

It throws out this cryptic error message:

HTTP Status 500 -

---

type Exception report

message

description The server encountered an internal error () that prevented it from fulfilling this request.

exception

javax.servlet.ServletException: Could not initialize class org.apache.commons.discovery.resource.names.DiscoverServiceNames
	org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:294)
	org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:162)
	org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:115)
	org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:433)
	org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
	org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
	org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:874)
	org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
	org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
	org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
	org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)
	java.lang.Thread.run(Thread.java:636)

root cause

java.lang.NoClassDefFoundError: Could not initialize class org.apache.commons.discovery.resource.names.DiscoverServiceNames
	org.kohsuke.stapler.Facet.discover(Facet.java:35)
	org.kohsuke.stapler.WebApp.<init>(WebApp.java:73)
	org.kohsuke.stapler.WebApp.get(WebApp.java:31)
	org.kohsuke.stapler.Stapler.init(Stapler.java:68)
	sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	java.lang.reflect.Method.invoke(Method.java:616)
	org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:244)
	java.security.AccessController.doPrivileged(Native Method)
	javax.security.auth.Subject.doAsPrivileged(Subject.java:537)
	org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:276)
	org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:162)
	org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:115)
	org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:433)
	org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
	org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:174)
	org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:874)
	org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
	org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
	org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
	org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689)

	java.lang.Thread.run(Thread.java:636)

note The full stack trace of the root cause is available in the Apache Tomcat/5.5 logs.

---

Apache Tomcat/5.5

Thankfully, a quick google search turned up this blog post.

http://testinfected.blogspot.com/2009/01/hudson-gets-accesscontrolexception-when.html

Hudson won’t work with Tomcat Security Manager enabled, but frankly, I can’t think of a realistic scenario where the security manager would do any good.  Does anyone allow cross-site servlet authoring?

“Here, enter some java code in this text box and we’ll compile and run it for you”  or “Click here to upload your war and I’ll deploy it.”

Does tomcat really allow this behavior by default?  (I guess I could see a place where it might be useful, such as appliances with embedded tomcat to push updates.)

workaround for bug in trac that wants to write to PYTHON_EGG_CACHE

I upgraded trac on a qa-site the other day, but then found I was getting “500 internal server error” about every other post.

Investigating revealed a log barf disguised as a stack trace, the final line of which was:

[Thu Mar 12 14:47:23 2009] [error] [client 24.16.139.248] PythonHandler trac.web.modpython_frontend: ExtractionError: Can’t extract file(s) to egg cache\n\nThe following error occurred while trying to extract file(s) to the Python egg\ncache:\n\n  [Errno 13] Permission denied: ‘/www/sites/fluffy/trac/.egg-cache’\n\nThe Python egg cache directory is currently set to:\n\n  /www/sites/fluffy/trac/.egg-cache\n\nPerhaps your account does not have write access to this directory?  You can\nchange the cache directory by setting the PYTHON_EGG_CACHE environment\nvariable to point to an accessible directory.\n, referer: http://fluffy.qa-site.com/trac/wiki/BuildFlexSdk?action=edit

I tried a simple solution, by creating the directory .egg-cache and giving apache write access.  But that’s no good.

Alternately, you could change the location of the environment variable PYTHON_EGG_CACHE.

You can do this with CGI, A bit better of a fix is to add the following line to your httpd.conf

SetEnv PYTHON_EGG_CACHE /tmp

Luckily, a lot of other people have seen this problem, and some of them even know a bit about mod_python

A helpful google search turned up a trac mailing list post gave me the hint I needed.

Thanks to Django’s funky setup, a lot of people have discovered that mod_python doesn’t take SetEnv.  I don’t know if this is a bug or a design flaw (perhaps with some justification).

Anyway here’s the hack for mod_python users, instead of SetEnv, use PythonOption:

PythonOption PYTHON_EGG_CACHE /tmp

MissionControl

Here’s an app under development that looks similar to my “QA Site” idea.

http://missioncontrolapp.com/

Prototypical QA Site Case Studies

Here are three hypothetical organizations that could benefit from One Shore consulting.

I’ll try to write these up into the website:

The following are composite potential customers and not real organizations.

QUICKER RAMP UP TIME, QA PROCESS IMPROVEMENT

Startup X  is growing rapidly.   Originally, the founders did all the coding and testing (as well as marketing and janitorial work) themselves, but now are busy managing the company and (window) shopping for private jets.

With new hires comes different coding styles and varied skillsets, and the original authors barely know the code base anymore (when they do have time to look into it.)  The last release slipped due to some last minute bugs discovered, and ongoing feature creep.   They need to introduce some discipline into the development process and ensure quality is maintained and deadlines don’t slip.

They know they need a test environment and a better build & deployment process, but don’t have the time and resources to do it themselves.

Proposed solution: a managed virtual test environment from One Shore.

A test lab is set up within one week.  No hardware needs purchased, no firewalls need penetrated, or permissions granted.  With every checkin, the new code is built and deployed automatically.   Smoke tests then run against the test environment and problems are detected immediately.  Releases run smoothly, and manual testing is quicker too, since a developer or tester doesn’t need to redeploy the whole project (and populate sample data) every time a change is made.

OPEN SOURCE AUTOMATION TOOLS EQUALS SAVINGS

Corporation Y is a large enterprise.  They have a rigorous testing process and use expensive proprietary tools.  Their license is expiring, and rather than renewing it they want to investigate using open source tools.

Some of the team members are advocates of open source and agile, but don’t know how best to persuade management that it’s safe.  A pilot project is proposed, and open source tools identified.  While they know what they want and know their proprietary products well, they don’t have the experience with the open source equivalents and don’t know the limitations.

Proposed solution: open source automation tool training and consulting by One Shore.

A report detailing the features and limitations of comparable open source tools tools is presented.  A workshop and some pair-programming helps SDETs quickly see how new tests can be written using the open source framework.   Migration of legacy tests can be outsourced to One Shore and reviewed internally by testers whose main focus can be on the new features.

ON DEMAND DOMAIN EXPERTISE NEEDED (OCCASIONALLY)

Company Z is not a “tech” business.  However, they do have a small in-houce IT staff that does occasional updates to their custom software application.  Because it is such a specialized field, it took a lot of time to train their tester, and they were reluctant to let her go, but with sometimes several months between releases, they didn’t see any alternative.

They tried staffing agencies, but besides the training time, the search for a qualified tester took an inordinate amount of time as well.  What they really need is someone who can hit the ground running, know their product, provides reliable results, and is willing to work only one month in three, part time.

Proposed solution: part time staffing from One Shore.

While there will still be the initial training time, consulting is what we do. Having multiple clients allows us to give as much (0r as little) time as needed to a client, and have top notch staff willing to work part time on your project.  Because of the variety of experiences, best practices are assured, and because we’ve worked with you in the past, there’s no headhunting and retraining time waste.

Three types of tests

The old saying goes something like this:

There are three types of people in the world, those that agree with me, those that disagree with me, and those that will see the error of their ways (and switch camps.)

Of course, the moral of that often misquoted koan is that arbitrary groupings are almost as tedious as their mathematical underpinnings in set theory.  So, for the sakes of the insomniacs reading my One Shore blog, I offer the following sedative meta-semantic rambling:

There are three types of tests:  validation tests, regression tests, and exploratory tests.

What follows is my definitions of the three and some weak arguments for my particular arbitrary grouping.

Validation testing ensures that a product meets the requirements set forth in the design.  (Not all requirements are explicit — nor are all designs complete.)  Functional testing frequently falls into this category.  So too, does unit testing, which is really code level functional testing (or not actually testing at all — see my other posts on unit testing.)

Regression testing verifies that nothing has broken (or re-broken) when changes are made.  Smoke tests fall into this category, but can also include (full or partial) functional (manual or automated)  regression testing.  Regression tests can either verify that functional previous validated hasn’t broken or that bugs are not re-introduced (“have not regressed“).

Exploratory testing The idea is to find out what you can do, what you can’t do, and what you can do that you shouldn’t be able to.  You’re typical ad-hoc testing is exploratory.  Exploratory doesn’t necessarily mean ad-hoc.  It can be targeted or methodical.  Usability testing is often exploratory in nature.  Security and stability are best ensured through exploratory testing.   The key is that you’re not testing that the requirements have been met, or that known defects have not regressed…you’re exploring.

These groupings are more about what your goal is than how you test.  You can see some bleed over and contradiction in my trying to pidgeonhole traditional test groupings into my newer (and correcter) categories.

All three types of tests are important, and the there should be a balance of all three in any project.  I don’t want to say they’re equally important, because the emphasis depends on the project.

Some projects might have very specify requirements that it is imperative are met.  Validation would thus carry more weight.   An existing code base may require thorough regression after even a small change.  Exploratory testing allows for intuition, inference, and even chance to play a factor in testing.

Exploratory testing finds the most bugs, hands down.  And it also generated the most feature requests.  Which is maybe why it’s not often a favorite of management. The simple reason is that you cannot fully define the full scope inside a set of features or defects.   It allows the tester’s brain to “guess”, to infer requirements or intuit potential weak points.

Regression testing is preventative in nature.  A good set of automated regression tests are easily the best way maintain quality.  But exploratory tests and validation can be automated as well.  Random input generators are exploratory in nature.  Test first development is validation testing as regression testing.

And now even I need to take a nap.

QA Site competition (of a sort)

I noticed that JumpBox has hosted wikis (mediawiki, docuwiki, moinmoin), blogs (wordpress, movable type) project management (projectpier, trac, redmine), bug tracking (bugzilla, mantis), monitoring (nagios/cacti, zenoss) and version control (subversion) tools.

Which means that you could get a jumpbox (or combination of jumpboxes) and do what the core of a QA Site does. Which means, in one sense, that there’s competition, of a sort, but in another sense, that there’s a demand for these hosted applications.

They don’t have them all put together in one package, though they soon might, but they don’t have a dashboard and integration. That’s what a QA Site will provide, and it may very well be an open source wrapper project the way WAMP provides an Apache + Mysql + PHP development package for Windows. So I’m not too worried there.

Jumpbox may even have interest in hosting QA Site packages, with a version control, bug tracking, project management, documentation, (and possibly even continuous integration and test case management) tools. Or I might use JumpBoxes as a basis for QA Sites.

The value I’m proposing isn’t the installation and hosting of these apps, but the expertise in using them together. Hosting is sort of a loss leader, or easy entryway. I expect downloaded VMWare & Xen appliances to be a logical progressive step from hosting, or an alternative for more tech-savvy organizations, and on-site full installations to be the greater demand.

In truth I have some trepidation about getting burdened with too much hosting responsibility and being required to spend more time than I’d like administering installations, and less time developing testing solutions.

So is Jumpbox more of a competitor or potential collaborator? Or, does it really just help grow the open source QA tools pie?

wget in ubuntu

at least its called ‘wget’

apt-get install wget

inetd

it just gets better and better:

apt-get install inetutils-inetd

aparently a zealotry for it’s own sake forbids xinetd, so install neither!

enabling apache ssl on ubuntu

does the insanity never stop?

# a2enmod ssl
# a2ensite default-ssl

see http://ubuntuforums.org/archive/index.php/t-4466.html

getting ‘service’ and ‘chkconfig’ on Ubuntu

typing /etc/init.d/apache2 restart is too much work, and sloppy

yet another Ubuntu shortcoming. Fix it with an optional tool (because sysadmins aren’t supposed to use Ubunut):

# apt-get install sysvconfig
# service apache2 status
* Apache is running (pid 8914).

#service apache2 restart
* Restarting web server apache2
… waiting .

Running it through service gives you a bit more feedback than calling the init.d script directly.

See:
https://help.ubuntu.com/community/SwitchingToUbuntu/FromLinux/%20RedHatEnterpriseLinuxAndFedora

Chkconfig is a bit more difficult. The alternatives are:

# update-rc.d
# sysv-rc-conf

The latter needs installed:

# apt-get install sysv-rc-conf

The former is byzantine, and I’m not even sure how it works or if it does what I want. It’s not a drop-in replacement for chkconfig.

see http://ubuntuforums.org/archive/index.php/t-20583.html