Let's talk a bit …
Technical
Automatically pack your Plone instances (without zeo)
Jan 20th
At Affinitic, we have many Plone sites on our servers and we wanted to pack their ZODB automatically.
Our goal is of course to do it without any downtime. It’s easy when you have a zeoserver, but you cannot use a packing script when there isn’t :
zc.lockfile.LockError: Couldn't lock 'instance/var/filestorage/Data.fs.lock'
We decided to simply do a wget on the ZMI packing form.
Here is the command to pack your main database (7 days) :
wget --max-redirect 0 --post-data='submit=Pack&days:float=7.0' http://user:password@localhost:8080/Control_Panel/Database/main/manage_pack
We wrapped this in a script to get meaningful exit code and this is deployed via Puppet to have a cron for each and every Plone instances. Even the user that is used to access the instance is created via Puppet with :
bin/instance adduser user password
Alternative method, not used for deployment reasons and using a Python script : https://www.nathanvangheem.com/news/automatically-pack-the-zodb
If it can help anyone …
Beware of uppercase letters in your config files
Aug 12th
We got a surprise using a [theme:parameters] variable in the manifest.cfg of one of our Diazo theme.
We were defining a parameter like that :
isFrontPage = context/@@isFrontPage
and then we were using it in the rules :
<drop css:content="#footer-sitemap" if="$isFrontPage" />
But we got an error after having the Theme installed. And we found that the parameter we got in Theme control panel was “isfrontpage” and not “isFrontPage” so the “isFrontPage” parameter used in rules was undefined !
This is happening because plone.app.theming (as plone.resource, Products.GenericSetup, …) is using python’s ConfigParser to parse the manifest.cfg file (“from ConfigParser import ConfigParser“) and it does a lower() on the sections and variables that you put in your config files.
This is the same for RawConfigParser and SafeConfigParser.
So … don’t ever use camelcase there 
Google Chrome + Firebug Lite + z3c.form = Bad combo
Jan 9th
There is a bug that provokes a double execution of the Update method of z3c.form when you launch Firebug Lite in Google Chrome. Be careful when you are developping in that environment.
I guess that firebug lite triggers an undesirable event.
Remove broken portlets programmatically
Mar 27th
If you have specific broken portlets that needs to be removed from your Plone site, you will have to reinstall the product that contains your portlets, otherwise you will get this error :
Traceback (innermost last): Module ZPublisher.Publish, line 126, in publish Module ZPublisher.mapply, line 77, in mapply Module ZPublisher.Publish, line 46, in call_object Module plone.app.portlets.browser.kss, line 66, in delete_portlet Module zope.container.ordered, line 243, in __delitem__ Module zope.container.contained, line 647, in uncontained Module OFS.Uninstalled, line 45, in __getattr__ AttributeError: __parent__
But what if you just can’t find / reinstall the product ?
You could already remove the broken portlet through Plone UI, thanks to packages like collective.braveportletsmanager. Now you will be able to do it without any additional package thanks to this change in plone.app.portlet.
If you want to delete broken portlets programmatically, this is possible by disabling the error raised before your usual portlet removal function (see fixing_up in zope.container.contained) :
from zope.container import contained
contained.fixing_up = True
manager = getUtility(IPortletManager, name=u'plone.leftcolumn', context=portal)
assignments = getMultiAdapter((portal, manager), IPortletAssignmentMapping)
for portlet in assignments:
del assignments[portlet]
contained.fixing_up = False
We used this in a migration step for a customer.
Enjoy !
Filter menu using a grok view
Mar 4th
You want a menu to be seen only in certain conditions without creating a new “permission” stuff. You can use the filter attribute in your menuItem. You can access the request and context in it.
Simple example:
<menuItem
for="*"
title="My dinosaur menu"
description=""
permission="zope.Public"
action="@@mydinosaurpage"
menu="mainmenu"
filter="context/@@displayForDinosaursOnly"/>
Now we have to create a view that will be called by the filter attribute, the only working way I found was using a grok view (e.g. menufilter.py):
from five import grok
from Products.CMFCore.utils import getToolByName
from zope.interface import Interface
class DisplayForDinosaursOnly(grok.View):
"""
Menu using that class as filter will be displayed
for Dinosaurs roles
"""
grok.context(Interface)
grok.name('displayForDinosaursOnly')
def render(self):
pm = getToolByName(self.context, 'portal_membership')
roles = pm.getAuthenticatedMember().getRoles()
return 'Dinosaur' in roles
And finally register your grok view:
<grok:grok package="my.package.ns.menufilter" />
And voilà! Your dinosaur menu will only be seen by dinosaurs, the filter is working.
Python verbose in zope and scripts
Feb 6th
To set python verbose, you can simply use -v option.
python -v foo.py
But how to do it in zope environment? This does not work:
bin/instance fg -v
So to make it work, just edit the first line in executable script, example in bin/instance
#!/home/user/buildout/foo.package/bin/python -v
Of course works with other scripts like bin/test, etc…
Now you can get more debug info when needed!
Zope – how long does it take to serve a request?
Oct 27th
For the moment only a reverse proxy (apache, ngnix) or a browser can show you how long a request took to be processed.
For monitoring or debugging purpose you might want to know what zope instance is slow.
The default access zope log file give you such a log format:
127.0.0.1 - Anonymous [27/Oct/2010:16:44:04 +0100] "GET / HTTP/1.1" 200 1957 "" "Mozilla/5.0 (X11; U; Linux x86_64; fr; rv:1.9.2.11) Gecko/20101013 Ubuntu/10.10 (maverick) Firefox/3.6.11"
which doesn’t give you any information about how fast zope was to serve that request.
At the moment there is no way to add this number of seconds in the log format of zope 2. Unfortunately the zope http server (ZServer & medusa) cannot be easily overridden or monkey patched.
Here is a patch to apply on your zope 2 buildout to have such a log format:
127.0.0.1 0.008843 - Anonymous [27/Oct/2010:16:45:41 +0100] "GET / HTTP/1.1" 200 1957 "" "Mozilla/5.0 (X11; U; Linux x86_64; fr; rv:1.9.2.11) Gecko/20101013 Ubuntu/10.10 (maverick) Firefox/3.6.11"
You see here that it took 0.008843 seconds to compute the page on that zope instance.
Thanks to this number of seconds we have monitoring checks (using Nagios 3) and monitoring graph (using Cacti) of each instances within our zeo’s.
To apply the patch, install omelette in your buildout and run this command in the root of your buildout:
patch -p0 < responsetime-212.diff
It will patch the required methods in ZServer to give you the number of seconds the request took to be processed. A patch might sound (and is) hacky but that’s the easiest way to change such a low level module.
Maybe we should give the possiblility to have this in log format in Zope 2.1 / 2.13 default config ?
OpenOffice with buildouts
Jan 8th
Easy install the pyuno library with buildout
This recipe was started by Infrae, I have added the pyuno egg creation within the recipe. It is so nice to be able to share code so easily !
Download OpenOffice in a buildout might sound odd but
- You got used (and I think you are right) to isolate your python environment for each projects (e.g using buildout & virtualenv)
- You want to control OpenOffice components with your favourite python using the official OpenOffice library: pyuno
- If you already installed OpenOffice, you might not see how to link your favourite python with the pyuno library provided by the OpenOffice package of your favourite distribution
The python uno library (aka pyuno) is only delivered with OpenOffice. The library provides access to all OpenOffice UNO api which means:
- read and write doc, odt, rdf, xls and the most common file extension known in the different well known office suites
- generate nice pdf
- no xml parsing/transforms when you want to read odt & co
- fill in templates with content (e.g. pod)
- drawing shapes
- … (you might want to have a look at the OO developper guide for more informations even if there is lot’s of java examples )
The only major disadvantage is that it requires to connect to an openoffice process. This can be a problem if your sysadmin don’t want to install a real X server in a production environment but this disadvantage is quickly fixed if you install and use Xvfb
This recipe can
- download OpenOffice from any url (default to OpenOffice 2.3) and extract it in your buildout
- replace the default python interpreter delivered with OO with the one you are using in your buildout
- create an egg with pyuno and link it inside your buildout
More info about installation and usage of this recipe on pypi
Linux is only supported by now (and I don’t want to be sorry for not having a mac), if you want to fix this the code is here
Zope 3 dependencies in Zope 2 buildout
Dec 10th
Describe a buildout recipe to avoid fetching zope 3 libraries when installing eggs with zope 3 depedencies in a zope 2 buildout
I got tired removing zope.interface, zope.component, zope.deferredimport, zope.event, … of my eggs folder inside my buildout when installing package such as z3c.sqlalchemy in my Zope2 / Plone buildouts …
After the discussion on Zope3-users list and thanks to Jim lights, I wrote a really simple recipe which just create egg links to zope libraries in your develop-eggs so that setuptools can see that the dependencies are
already satisfied.
So if you list your zope library :
$ ls yourbuildout/parts/yourzope2/lib/python/zope app configuration documenttemplate exceptions i18n __init__.py pagetemplate schema structuredtext testbrowser viewlet cachedescriptors contentprovider dottedname formlib i18nmessageid interface proxy security tal testing component deprecation event hookable index modulealias publisher server tales thread
You will get
$ ls yourbuildout/develop-eggs zope.app.component.egg-info zope.app.security.egg-info zope.dottedname.egg-info zope.interface.egg-info zope.structuredtext.egg-info zope.app.egg-info zope.app.testing.egg-info zope.event.egg-info zope.modulealias.egg-info zope.tal.egg-info zope.app.event.egg-info zope.cachedescriptors.egg-info zope.exceptions.egg-info zope.pagetemplate.egg-info zope.tales.egg-info zope.app.i18n.egg-info zope.component.egg-info zope.formlib.egg-info zope.proxy.egg-info zope.testbrowser.egg-info zope.app.interface.egg-info zope.configuration.egg-info zope.hookable.egg-info zope.publisher.egg-info zope.testing.egg-info zope.app.pagetemplate.egg-info zope.contentprovider.egg-info zope.i18n.egg-info zope.schema.egg-info zope.thread.egg-info zope.app.publisher.egg-info zope.deprecation.egg-info zope.i18nmessageid.egg-info zope.security.egg-info zope.viewlet.egg-info zope.app.schema.egg-info zope.documenttemplate.egg-info zope.index.egg-info zope.server.egg-info
Where each of these file will be seen for setuptools as an egg:
$ cat yourbuildout/develop-eggs/zope.app.component.egg-info Metadata-Version: 1.0 Name: zope.app.component Version: 0.0
Sure this might look like an ugly hook but I can’t wait for zope 2 eggification …
Buildout and Virtualenv
Dec 6th
How and why use buildout with virtualenv
Buildout is this wonderful tool which helps you to automate setup and configuration of your applications.
Virtualenv is a tool which will help you to isolate your python environment
A few days ago I got stuck during a long time because I didn’t see that one library I installed in my global site-packages of my favourite python 2.4 (on my ubuntu: /usr/lib/python2.4/site-packages/) was a lower version of a library I was using in my buildout. Package was sqlalchemy 0.4 in my global sites-package and my buildout based application was using sqlalchemy 0.3.8 … Here is a simple solution to avoid this kind of things.
Idea is to start buildout with a python free of any external library. Virtualenv is the answer.
Install Virtualenv
Easy:
$ easy_install virtualenv
you will have then a file that you can run: /usr/bin/virtualenv
Let’s say you have a buildout configuration go into it:
$ cd myApp.buildout $ ls -l bootstrap.py buildout.cfg
Now you just have to create the python environment without any access to the global site-packages with virtualenv:
$ virtualenv --no-site-packages .
You will have then your new python in the bin folder :
$ ls -l bin activate easy_install easy_install-2.4 python2.4
You can now run the buildout configuration with your new python:
$ ./bin/python2.4 bootstrap.py $ ./bin/buildout
This will create you a nice and totally isolated environment …