No more JS / CSS browser cache refresh pain

We often have the following problem :

  1. a CSS or a Javascript file is changed in a package
  2. we make a release (or not) and put it to production
  3. some visitors still gets old CSS and Javascript versions (except if by chance we remembered to reinstall package / re-save portal_css and portal_javascripts configs) and need to flush their browser cache

As we do not have great memory and as our customer’s Helpdesk was tired of doing remote cache flush, we wanted to do something (automatic) about it.
Plone adds cache keys as suffixes for CSS and JS files, as for example base-cachekey6247.css.
This cache key doesn’t change, even after an instance restart. This is why browsers can keep the “old” versions cached.

To avoid that (and without touching to Apache or whatever), we wrote a little script that exectutes at the end of the instance startup process.It forces the “cook” of resources which generates a new cache key so that browser cannot cache CSS and JS anymore !

This script can of course be improved, but it fits our needs.

[Please note that this code is written for Plone 3.]
First, we subscribe to IDatabaseOpenedWithRootEvent (too bad we couldn’t use IProcessStarting because we have no context to get Plone sites) :

  <subscriber                                                               
      for="zope.app.appsetup.interfaces.IDatabaseOpenedWithRootEvent"
      handler=".resources.clear_cache"                               
      />

Then, our handler gets the work done :

# -*- coding: utf-8 -*-                                                    

from Products.CMFCore.utils import getToolByName                           
from zope.app.appsetup.bootstrap import getInformationFromEvent            
import transaction                                                         

from our.package import logger                                       

def clear_cache(event):                                                    
    """                                                     
    Force cache key renewal for CSS and JS on all Plone sites at startup.  
    This avoids any caching of changed CSS or JS.          
    """                                                     
    db, connection, root, root_folder = getInformationFromEvent(event)     
    app = root_folder                                                      
    for obj_tuple in app.items():                                           
        (obj_name, obj) = obj_tuple                                            
        if obj.get('portal_type') == 'Plone Site':                         
            p_css = getToolByName(obj, 'portal_css')                        
            p_css.cookResources()                                           
            p_js = getToolByName(obj, 'portal_javascripts')                 
            p_js.cookResources()                                            
            logger.info('Cache key changed for CSS/JS on Plone %s' % obj_name)
    transaction.commit()

And voilĂ , no more browser cache flush hassle for our customer and their Helpdesk :-)

If you have any thoughts on this, or think that this should be packaged in a collective egg (useful for you ?), feel free to comment.

Get rid of huge images in your Plone site

Do you have users that don’t care about image sizes at all ?
Do you have Plone sites with tons of raw photos galleries ?
Do you want to shrink your Data.fs ?

Then try the new collective.autoscaling package !

It allows automatic scaling of large images uploaded into your Plone sites.
You can configure the maximum width / height you are willing to tolerate and it does the rest !
This is totally transparent to the user (except if you choose to show message).

Autoscaling settings

Images can be either Image content type or any Image field on Dexterity content types.

Example use case :

  1. You configure collective.autoscaling to have images with maximum size of height 800px / width 1200px.
  2. One of your user uploads a really big image : height 2000px / width 4000px.
  3. This image will be resized to height 600px / width 1200px (aspect ratio is of course preserved).

There is also a migration view you can call on any folder (or at the site root) to resize all contained images.

 

collective.autoscaling (developed for IMIO) is available on Github and version 1.0 has just been released on Pypi.

Please enjoy, improve and send feedback !

 

Automatically pack your Plone instances (without zeo)

At Affinitic, we have many Plone sites on our servers and we wanted to pack their ZODB automatically.

Our goal is of course to do it without any downtime. It’s easy when you have a zeoserver, but you cannot use a packing script when there isn’t :

zc.lockfile.LockError: Couldn't lock 'instance/var/filestorage/Data.fs.lock'

We decided to simply do a wget on the ZMI packing form.
Here is the command to pack your main database (7 days) :

wget --max-redirect 0 --post-data='submit=Pack&days:float=7.0' http://user:password@localhost:8080/Control_Panel/Database/main/manage_pack

We wrapped this in a script to get meaningful exit code and this is deployed via Puppet to have a cron for each and every Plone instances. Even the user that is used to access the instance is created via Puppet with :

bin/instance adduser user password

Alternative method, not used for deployment reasons and using a Python script : https://www.nathanvangheem.com/news/automatically-pack-the-zodb

If it can help anyone …

Omit attribute in TAL

I always tough than it was impossible to completely omit a tag attribute using TAL. So when I wanted to automatically check or not a radio button, I duplicated it like this:

<input type="radio" tal:condition="mycondition" checked="checked" />
<input type="radio" tal:condition="not:mycondition" />

But we can omit a tag attribute by passing nothing to a tal:attributes, like this:

<input type="radio" tal:attributes="checked python:mycondition and 'checked' or nothing" />

It’s so cleaner!

Keyword fields are tied to allowRolesToAddKeywords roles

For a project, we have added some extended fields on contents. Those fields use KeywordWidget widget (same as Plone ‘Subject’ tags).

We wanted to have a custom permission on those fields, so we used :

read_permission=permissions.OurCustomPermission
write_permission=permissions.OurCustomPermission

But it didn’t work : the field title and label displayed correctly (when we have the permission) but the widget never happeared.

Why ?

Because the KeywordWidget template use allowRolesToAddKeywords (from portal_properties / site_properties) to filter who can add new keywords. So that “permission” will impact your custom fields too !

There has been tickets / discussion about allowRolesToAddKeywords, but that never changed.

Hope that noone else would get bitten by this once-for-all permission on keyword widgets.

Affinitic at Monster GameJam

We participated in the Monster GameJam contest, the deal was to create a video game within only 2 days.
The given subject was “Inner mechanism”, so we had the idea to develop a game about an old guy trying to recursively explore his memory. It is a puzzle/memory game playable with arrow keys only.
You can play at our game here MemoryKey, and at all other participants projects itch.io Monster GameJam.

Our game and most of the others were developped using the great game dev framework Unity. If you are interested in game development, you have to try it out.

At the event, we were interviewed by DHnet.be, they wanted to write an article about the lack of independent video game developpers in Belgium.

Check the nice geek T-Shirts we were wearing and how concentrated we were …

gamejam_team2

The game is over!

gamejam_team

And everyone play to everyone’s games.

gamejam_demo

Beware of uppercase letters in your config files

We got a surprise using a [theme:parameters] variable in the manifest.cfg of one of our Diazo theme.

We were defining a parameter like that :

isFrontPage = context/@@isFrontPage

and then we were using it in the rules :

<drop css:content="#footer-sitemap" if="$isFrontPage" />

But we got an error after having the Theme installed. And we found that the parameter we got in Theme control panel was “isfrontpage” and not “isFrontPage” so the “isFrontPage” parameter used in rules was undefined !

This is happening because plone.app.theming (as plone.resource, Products.GenericSetup, …) is using python’s ConfigParser to parse the manifest.cfg file (“from ConfigParser import ConfigParser“) and it does a lower() on the sections and variables that you put in your config files.

This is the same for RawConfigParser and SafeConfigParser.

So … don’t ever use camelcase there ;-)

 

Google Chrome + Firebug Lite + z3c.form = Bad combo

There is a bug that provokes a double execution of the Update method of z3c.form when you launch Firebug Lite in Google Chrome. Be careful when you are developping in that environment.

I guess that firebug lite triggers an undesirable event.

Percent in python’s string formatting

In python, we don’t need to escape the char ‘%’ in a string, except when we want to format the specific string. Examples:

>>> 'Here is a percent: %'
'Here is a percent: %'

>>> 'Here is two percent %%'
'Here is two percent %%'

>>> 'Here is a percent %% %s' % 'in formatted string'
'Here is a percent % in formatted string'

>>> 'We have to double the percent % %s!' % 'in formatted string'
Traceback (most recent call last):
  File "", line 1, in 
TypeError: not all arguments converted during string formatting

>>> 'If we do not specify a format, we must not double the percent: %(foo)s %(bar)s!'
'If we do not specify a format, we must not double the percent: %(foo)s %(bar)s!'

>>> 'If we want to format specifically, we must double the percent: %(foo)s %%(bar)s!' % {'foo': 'test'}
'If we want to format specifically, we must double the percent: test %(bar)s!'

>>> 'And of course this will crash: %(foo)s %(bar)s!' % {'foo': 'test'}
Traceback (most recent call last):
  File "", line 1, in 
KeyError: 'bar'

New package affinitic.templer

We just created a new package to gain a lot of time when we create new plone packages from scratch. The main goal is to maximum simplify this creation. This can include a diazo theme.

All the details are in the README of the package found in https://github.com/affinitic/templer.affinitic