Remove broken portlets programmatically

If you have specific broken portlets that needs to be removed from your Plone site, you will have to reinstall the product that contains your portlets, otherwise you will get this error :

Traceback (innermost last):
Module ZPublisher.Publish, line 126, in publish
Module ZPublisher.mapply, line 77, in mapply
Module ZPublisher.Publish, line 46, in call_object
Module plone.app.portlets.browser.kss, line 66, in delete_portlet
Module zope.container.ordered, line 243, in __delitem__
Module zope.container.contained, line 647, in uncontained
Module OFS.Uninstalled, line 45, in __getattr__
AttributeError: __parent__

But what if you just can’t find / reinstall the product ?

You could already remove the broken portlet through Plone UI, thanks to packages like collective.braveportletsmanager. Now you will be able to do it without any additional package thanks to this change in plone.app.portlet.

If you want to delete broken portlets programmatically, this is possible by disabling the error raised before your usual portlet removal function (see fixing_up in zope.container.contained) :

from zope.container import contained
contained.fixing_up = True
manager = getUtility(IPortletManager, name=u'plone.leftcolumn', context=portal)
assignments = getMultiAdapter((portal, manager), IPortletAssignmentMapping)
for portlet in assignments:
    del assignments[portlet]
contained.fixing_up = False

We used this in a migration step for a customer.
Enjoy !

Filter menu using a grok view

You want a menu to be seen only in certain conditions without creating a new “permission” stuff. You can use the filter attribute in your menuItem. You can access the request and context in it.

Simple example:

<menuItem
    for="*"
    title="My dinosaur menu"
    description=""
    permission="zope.Public"
    action="@@mydinosaurpage"
    menu="mainmenu"
    filter="context/@@displayForDinosaursOnly"/>

Now we have to create a view that will be called by the filter attribute, the only working way I found was using a grok view (e.g. menufilter.py):

from five import grok
from Products.CMFCore.utils import getToolByName

from zope.interface import Interface

class DisplayForDinosaursOnly(grok.View):
    """
    Menu using that class as filter will be displayed
        for Dinosaurs roles
    """
    grok.context(Interface)
    grok.name('displayForDinosaursOnly')

    def render(self):
        pm = getToolByName(self.context, 'portal_membership')
        roles = pm.getAuthenticatedMember().getRoles()
        return 'Dinosaur' in roles

And finally register your grok view:

<grok:grok package="my.package.ns.menufilter" />

And voilà! Your dinosaur menu will only be seen by dinosaurs, the filter is working.

Python verbose in zope and scripts

To set python verbose, you can simply use -v option.

python -v foo.py

But how to do it in zope environment? This does not work:

bin/instance fg -v

So to make it work, just edit the first line in executable script, example in bin/instance

#!/home/user/buildout/foo.package/bin/python -v

Of course works with other scripts like bin/test, etc…

Now you can get more debug info when needed!

Comment traduire des contenus dans Plone 4

How to translate Plone 4 content – French manual

I. Accéder à l’édition de contenus

Pour accéder à l’édition des contenus du site Plone et à l’interface de traduction, il faut se connecter avec son identifiant et son mot de passe via le lien “Se connecter” situé en haut à droite du site.

Se connecter

En règle générale, il est important de toujours créer un nouveau contenu d’abord dans la langue de base du site (ici le français). Si ce n’est pas le cas, on obtient rapidement des incohérences ou des fonctionnements non homogènes au niveau des traductions, et il devient donc très difficile de les maintenir.

La langue de base du site est fixée lors de la création du site.

II. Traduction d’un contenu

Si le contenu existe déjà (en français)

Pour les éléments déjà présents sur le site et qui n’existent que dans la langue de base (FR), voici la procédure à appliquer pour les traduire :

1. après s’être connecté, il faut vérifier que l’on se trouve bien dans la langue de base (par défaut) du site. Dans le cas de cet article, cette langue de base est le français.

English Francais

2. naviguer vers le contenu (francophone) à traduire

3. cliquer sur l’onglet “traduire en …” et sélectionner la langue cible (dans cet exemple, l’anglais)

Traduire en

4. il suffit alors de traduire les informations du contenu de base, champ par champ. La page se compose de deux colonnes : à gauche la langue de base (FR) et à droite les champs à éditer dans la langue cible (EN). L’édition de la traduction est donc grandement facilitée par la vision du contenu de base.

Traductions en colonnes

Une fois le contenu intégré dans les champs correspondants, il suffit de cliquer sur “sauver” pour enregistrer la nouvelle traduction.

Attention : accéder à cet écran de traduction pour une langue crée la traduction même si son édition est annulée (via le bouton “Annuler”).

En outre, par défaut, la traduction se trouve en statut privé. Elle peut donc être complétée en plusieurs fois et, une fois son édition complètement terminée, il faudra la publier pour qu’elle devienne publique et visible par tous les internautes. Pour cela, il faut changer l’état du contenu, de “Privé” à “Publié” (“Private” en “Published”).

Publier

Si le contenu n’existe pas encore (du tout)

Comme expliqué plus haut, si l’on désire ajouter un nouvel élément de contenu, il est important de toujours le faire dans la langue de base du site (FR dans le cas présent), via le menu “Ajout d’un élément”.

Ajout d'un élément

Une fois l’élément de contenu créé et sauvé, il devient possible de le traduire dans les différentes autres langues du site.

Cette procédure est alors la même que celle de l’étape précédente (traduction de contenu existant).

III. Modifier une traduction existante

Pour éditer une traduction déjà réalisée, il existe deux façons de faire :

1. accéder au contenu – dont la traduction doit être modifiée – dans la langue de base du site (FR). Ensuite, changer la langue du site pour la langue cible.

Changement de langue

On se retrouvera alors sur la traduction du contenu, et un simple clic sur “Modifier” amènera à la page d’édition de traduction (langue de base à gauche, champs de traduction à droite).

2. naviguer sur le site – directement dans la langue cible – vers la traduction à éditer. Une fois le contenu atteint, il suffit de cliquer sur “Modifier” (“Edit” en EN), et on se retrouve alors sur la page de traduction contenant les deux colonnes de champs.

Zope – how long does it take to serve a request?

For the moment only a reverse proxy (apache, ngnix) or a browser can show you how long a request took to be processed.

For monitoring or debugging purpose you might want to know what zope instance is slow.

The default access zope log file give you such a log format:

127.0.0.1 - Anonymous [27/Oct/2010:16:44:04 +0100] "GET / HTTP/1.1" 200 1957 "" "Mozilla/5.0 (X11; U; Linux x86_64; fr; rv:1.9.2.11) Gecko/20101013 Ubuntu/10.10 (maverick) Firefox/3.6.11"

which doesn’t give you any information about how fast zope was to serve that request.

At the moment there is no way to add this number of seconds in the log format of zope 2. Unfortunately the zope http server (ZServer & medusa) cannot be easily overridden or monkey patched.

Here is a patch to apply on your zope 2 buildout to have such a log format:

127.0.0.1 0.008843 - Anonymous [27/Oct/2010:16:45:41 +0100] "GET / HTTP/1.1" 200 1957 "" "Mozilla/5.0 (X11; U; Linux x86_64; fr; rv:1.9.2.11) Gecko/20101013 Ubuntu/10.10 (maverick) Firefox/3.6.11"

You see here that it took 0.008843 seconds to compute the page on that zope instance.

Thanks to this number of seconds we have monitoring checks (using Nagios 3) and monitoring graph (using Cacti) of each instances within our zeo’s.

Patch for Zope 2.10

Patch for Zope 2.12

To apply the patch, install omelette in your buildout and run this command in the root of your buildout:

patch -p0 < responsetime-212.diff

It will patch the required methods in ZServer to give you the number of seconds the request took to be processed. A patch might sound (and is) hacky but that’s the easiest way to change such a low level module.

Maybe we should give the possiblility to have this in log format in Zope 2.1 / 2.13 default config ?

OpenOffice with buildouts

Easy install the pyuno library with buildout

This recipe was started by Infrae, I have added the pyuno egg creation within the recipe. It is so nice to be able to share code so easily !

Download OpenOffice in a buildout might sound odd but

  • You got used (and I think you are right) to isolate your python environment for each projects (e.g using buildout & virtualenv)
  • You want to control OpenOffice components with your favourite python using the official OpenOffice library: pyuno
  • If you already installed OpenOffice, you might not see how to link your favourite python with the pyuno library provided by the OpenOffice package of your favourite distribution

The python uno library (aka pyuno) is only delivered with OpenOffice. The library provides access to all OpenOffice UNO api which means:

  • read and write doc, odt, rdf, xls and the most common file extension known in the different well known office suites
  • generate nice pdf
  • no xml parsing/transforms when you want to read odt & co
  • fill in templates with content (e.g. pod)
  • drawing shapes
  • … (you might want to have a look at the OO developper guide for more informations even if there is lot’s of java examples )

The only major disadvantage is that it requires to connect to an openoffice process. This can be a problem if your sysadmin don’t want to install a real X server in a production environment but this disadvantage is quickly fixed if you install and use Xvfb

This recipe can

  • download OpenOffice from any url (default to OpenOffice 2.3) and extract it in your buildout
  • replace the default python interpreter delivered with OO with the one you are using in your buildout
  • create an egg with pyuno and link it inside your buildout

More info about installation and usage of this recipe on pypi

Linux is only supported by now (and I don’t want to be sorry for not having a mac), if you want to fix this the code is here ;)

Zope 3 dependencies in Zope 2 buildout

Describe a buildout recipe to avoid fetching zope 3 libraries when installing eggs with zope 3 depedencies in a zope 2 buildout

I got tired removing zope.interface, zope.component, zope.deferredimport, zope.event, … of my eggs folder inside my buildout when installing package such as z3c.sqlalchemy in my Zope2 / Plone buildouts …

After the discussion on Zope3-users list and thanks to Jim lights, I wrote a really simple recipe which just create egg links to zope libraries in your develop-eggs so that setuptools can see that the dependencies are
already satisfied.

So if you list your zope library :

$ ls yourbuildout/parts/yourzope2/lib/python/zope

app               configuration    documenttemplate  exceptions  i18n           __init__.py  pagetemplate  schema    structuredtext  testbrowser  viewlet
cachedescriptors  contentprovider  dottedname        formlib     i18nmessageid  interface    proxy         security  tal             testing
component         deprecation      event             hookable    index          modulealias  publisher     server    tales           thread

You will get

$ ls yourbuildout/develop-eggs

zope.app.component.egg-info     zope.app.security.egg-info      zope.dottedname.egg-info     zope.interface.egg-info     zope.structuredtext.egg-info
zope.app.egg-info               zope.app.testing.egg-info       zope.event.egg-info          zope.modulealias.egg-info   zope.tal.egg-info
zope.app.event.egg-info         zope.cachedescriptors.egg-info  zope.exceptions.egg-info     zope.pagetemplate.egg-info  zope.tales.egg-info
zope.app.i18n.egg-info          zope.component.egg-info         zope.formlib.egg-info        zope.proxy.egg-info         zope.testbrowser.egg-info
zope.app.interface.egg-info     zope.configuration.egg-info     zope.hookable.egg-info       zope.publisher.egg-info     zope.testing.egg-info
zope.app.pagetemplate.egg-info  zope.contentprovider.egg-info   zope.i18n.egg-info           zope.schema.egg-info        zope.thread.egg-info
zope.app.publisher.egg-info     zope.deprecation.egg-info       zope.i18nmessageid.egg-info  zope.security.egg-info      zope.viewlet.egg-info
zope.app.schema.egg-info        zope.documenttemplate.egg-info  zope.index.egg-info          zope.server.egg-info

Where each of these file will be seen for setuptools as an egg:

$ cat yourbuildout/develop-eggs/zope.app.component.egg-info

Metadata-Version: 1.0
Name: zope.app.component
Version: 0.0

Sure this might look like an ugly hook but I can’t wait for zope 2 eggification …

Buildout and Virtualenv

How and why use buildout with virtualenv

Buildout is this wonderful tool which helps you to automate setup and configuration of your applications.

Virtualenv is a tool which will help you to isolate your python environment

A few days ago I got stuck during a long time because I didn’t see that one library I installed in my global site-packages of my favourite python 2.4 (on my ubuntu: /usr/lib/python2.4/site-packages/) was a lower version of a library I was using in my buildout. Package was sqlalchemy 0.4 in my global sites-package and my buildout based application was using sqlalchemy 0.3.8 … Here is a simple solution to avoid this kind of things.

Idea is to start buildout with a python free of any external library. Virtualenv is the answer.

Install Virtualenv

Easy:

$ easy_install virtualenv

you will have then a file that you can run: /usr/bin/virtualenv

Let’s say you have a buildout configuration go into it:

$ cd myApp.buildout
$ ls -l
bootstrap.py
buildout.cfg

Now you just have to create the python environment without any access to the global site-packages with virtualenv:

$ virtualenv --no-site-packages .

You will have then your new python in the bin folder :

$ ls -l bin
activate
easy_install
easy_install-2.4
python2.4

You can now run the buildout configuration with your new python:

$ ./bin/python2.4 bootstrap.py
$ ./bin/buildout

This will create you a nice and totally isolated environment …

Writing nicer Python code

Starting from the basic facts:

  • you can’t always code in pair
  • you don’t want to compile & tests to check that is fine in the last few line you have just written
  • you want to write nice python code
  • you want to improve the way you write python

It exists a few very interesting softwares that can help you.

Here is a list of the one I am using really often:

PyFlakes – http://www.divmod.org/projects/pyflakes

PyLint – http://www.logilab.org/view?rql=Any%20X%20WHERE%20X%20eid%20857

These two are nice also but not as good as the previous one:

PyMetrics – http://sourceforge.net/projects/pymetrics

PyChecker – http://pychecker.sourceforge.net/

These tool are directly linked to my favourite editor VI:

command Pyflakes :call Pyflakes()
function! Pyflakes()
    let tmpfile = tempname()
    execute "w" tmpfile
    execute "set makeprg=(pyflakes\\ " . tmpfile . "\\\\\\|sed\\ s@" . tmpfile ."@%@)"
    make
    cw
endfunction
command Pylint :call Pylint()
function! Pylint()
    setlocal makeprg=(echo\ '[%]';\ pylint\ %)
    setlocal efm=%+P[%f],%t:\ %#%l:%m
    silent make
    cwindow
    endfunction

And better each time you save your python file in vim , I check for wrong imports with Pyflakes with:

autocmd BufWrite *.{py} :call Pyflakes()

Watch my new commit…

Get notification as soon as somebody commit is really important! A quick note to set it up with emails…

We will make this in 4 steps.

Step 1: Installing pysvn

Check that you don’t have it yet:

$ python2.4
Python 2.4.3 (#2, Oct  6 2006, 07:52:30)
[GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import svn
>>>

If you get something else (e.g ImportError: No module named svn) you will need to do this step:

You will need the python library to access subversion: pysvn. Note that header of the libsvn are required [should come with your subversion install]!
Go to http://pysvn.tigris.org/project_downloads.html and fetch last tarball (MacOSX version also available). Note that
pysvn is noted there as “Extension”. I fetched last stable version: http://pysvn.tigris.org/files/documents/1233/34994/pysvn-1.5.0.tar.gz

$ tar xvzf pysvn-1.5.0.tar.gz
 ...
$ cd pysvn-1.5.0/Source
$ python2.4 setup.py configure
...
$ make
...[pray]...
$ mkdir /usr/lib/python2.4/site-packages/pysvn
$ cp pysvn/__init__.py /usr/lib/python2.4/site-packages/pysvn
$ cp pysvn/_pysvn.so /usr/lib/python2.4/site-packages/pysvn

Now try again and you should get something like:

$ python2.4
Python 2.4.3 (#2, Oct  6 2006, 07:52:30)
[GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import svn
>>>


Step 2: Installing SVNMailer


Go to http://storage.perlig.de/svnmailer/ and fetch last tarball (stable actual one is: http://storage.perlig.de/svnmailer/svnmailer-1.0.8.tar.gz)

Fallow these steps:

$ tar xvzf svnmailer-1.0.8.tar.gz
   ...

$ cd svnmailer-1.0.8
$ /usr/bin/python2.4 setup.py install
   ...

You have now a brand new svnmailer installed. Check it with:

$ ls -l /usr/bin/svnmailer
-rwxr-xr-x 1 root root 2192 2006-10-24 17:00 /usr/bin/svn-mailer


Step 3: Configure your repository to use svnmailer

We created a svn repository:

$ svnadmin create /var/svn/repos1

So we have a repository on our local filesystem in /var/svn/repos1


Go into the hooks directory:

$ cd /var/svn/repos1/hooks

Add/edit post-commit file.
And add the fallowing lines:

#!/bin/sh
REPOS="$1"
REV="$2"
/usr/bin/svn-mailer commit "$REPOS" "$REV" /etc/svn-mailer.conf

Change execute permission on post-commit

chmod 755 post-commit

It’s time to configure SVNMailer

Step 4: Configure SVNMailer

Edit/Add the file /etc/svn-mailer.conf
And add the fallowing lines [you might edit few things...]. Imagine that I have mymodule in my repository (so that i can do svn co file:///var/svn/repos1/mymodule)

[general]
# see http://opensource.perlig.de/svnmailer/doc-1.0/#general for details.

# the diff command to be used ... just copy it...
diff = /usr/bin/diff -u -L %(label_from)s -L %(label_to)s %(from)s %(to)s
# the sendmail location
mail_command = /usr/sbin/sendmail

[mymodule]
# see http://opensource.perlig.de/svnmailer/doc-1.0/#groups for details
# this part of the config apply only for commits under mymodule
for_paths = mymodule/.*

# the subject of the email
commit_subject_prefix = [MYMODULE]

# From address in the mail
from_addr = jfroche@jfroche.be

# To address ...
to_addr = peopleinterestinginmymodule@foo.bar

[defaults]
# see http://opensource.perlig.de/svnmailer/doc-1.0/#groups for details
# this part of the config apply for all the other module

# Default From address template
from_addr = %(author)s@localhost.localdomain

# the subject of the email
commit_subject_prefix = [SVN]
to_addr = jeff@jfroche.be foo@skynet.be bar@gmail.com
generate_diffs = add copy modify
suppress_deletes = yes

Here it is… Try to commit and you should get email…

If you want to traceback error, go to edit /var/svn/repos1/hooks/post-commit and for example log to a file by changing the line

/usr/bin/svn-mailer commit "$REPOS" "$REV" /etc/svn-mailer.conf

In

/usr/bin/svn-mailer commit "$REPOS" "$REV" /etc/svn-mailer.conf 2> 1> /tmp/svnmailer.log

Hope this helps …