[3296] | 1 | A cachefu with less monkeypatching |
---|
| 2 | ================================== |
---|
| 3 | |
---|
| 4 | Current patching in cachefu |
---|
| 5 | --------------------------- |
---|
| 6 | |
---|
| 7 | Currently, cachefu does a lot of monkeypatching. This is a list of the |
---|
| 8 | files in which monkeypatching takes place: |
---|
| 9 | |
---|
| 10 | CacheSetup/patch.py - |
---|
| 11 | OFS.Image/File are patched so that they're associated with the |
---|
| 12 | PolicyHTTPCacheManager to get control over old-style files and |
---|
| 13 | images. |
---|
| 14 | |
---|
| 15 | A ``.modified()`` method is patched into old-style files and images |
---|
| 16 | and into filesystem files and images to support asking them about a |
---|
| 17 | last modification date. |
---|
| 18 | |
---|
| 19 | The catalog is patched: ``moveObjectsByDelta()``, |
---|
| 20 | ``uncatalog_object()`` and ``catalog_object()``. All now call a |
---|
| 21 | method that purges squid (except for uncatalog_object) so that the |
---|
| 22 | object in squid gets purged whenever it changes. "Changes" in this |
---|
| 23 | case is defined as "when the catalog changes". It was a good way to |
---|
| 24 | detect it for older plones, but at the moment not everything is |
---|
| 25 | detected this way. IObjectModified event to the resque? |
---|
| 26 | |
---|
| 27 | The squid purge method also increments a counter (also for |
---|
| 28 | uncatalog_object) inside the cache tool: this is used as a sort |
---|
| 29 | last-modified-date for the catalog. This counter is used quite a lot |
---|
| 30 | by cachefu to keep everything updated, especially for logged in |
---|
| 31 | users. |
---|
| 32 | |
---|
| 33 | ResourceRegistries' ``cookResources()`` is patched so that |
---|
| 34 | abovementioned catalog counter gets incremented everything something |
---|
| 35 | changes inside ResourceRegistries. |
---|
| 36 | |
---|
| 37 | ``manage_changePermissions()``, ``manage_permission()``, |
---|
| 38 | ``manage_acquiredPermissions()`` and ``manage_role()`` are patched |
---|
| 39 | to increment a counter every time the relationship between |
---|
| 40 | permissions and roles change. This counter can then be used to |
---|
| 41 | determine freshness. |
---|
| 42 | |
---|
| 43 | CacheSetup/patch_cmf.py - |
---|
| 44 | ``FSPageTemplate`` gets patched so that the filesystem (=skin |
---|
| 45 | directory) template doesn't get rendered if a "304 not modified" can |
---|
| 46 | be returned. Rendering the template is expensive and it isn't needed |
---|
| 47 | if a 304 can be returned. Secondly, cache headers are set after the |
---|
| 48 | page template has been rendered. |
---|
| 49 | |
---|
| 50 | The pagetemplate rendering mechanism, ``TALInterpreter``, is patched |
---|
| 51 | likewise. 304 interruption and cache heading setting. |
---|
| 52 | |
---|
| 53 | CacheSetup/patch_interpreter.py - |
---|
| 54 | This is a patch to enable macro caching. It looks like it only works |
---|
| 55 | with plone 2.5 (not 3.0) at the moment because of the import of the |
---|
| 56 | TAL.TALInterpreter, which isn't available anymore in plone 3.0's |
---|
| 57 | zope version. |
---|
| 58 | |
---|
| 59 | Many of the people at the sprint didn't seem to like macro caching, |
---|
| 60 | so it might be best to just ignore it for the time being. In that |
---|
| 61 | case the patch can stay, but will only be useful for 2.5. |
---|
| 62 | |
---|
| 63 | Note: We've indeed removed it in trunk (i.e. for 3.0) on the PIKtipi |
---|
| 64 | Sprint (tomster, fschulze) |
---|
| 65 | |
---|
| 66 | CacheSetup/patch_utils.py - |
---|
| 67 | Actually, this is just a utility class that provides some handy |
---|
| 68 | methods for patching files. |
---|
| 69 | |
---|
| 70 | CMFSquidTool/patch.py - |
---|
| 71 | Just a utility class for patching. What's the possibility for |
---|
| 72 | merging it with above one? |
---|
| 73 | |
---|
| 74 | CMFSquidTool/queue.py - |
---|
| 75 | ``reindexObject()`` and ``unindexObject()`` of CMFCatalogAware and |
---|
| 76 | CatalogMultiplex are patched to put objects in the squid purge |
---|
| 77 | queue. Didn't ``CacheSetup/patch.py`` already to that? Might be a |
---|
| 78 | chance for some serious code cleanup here. |
---|
| 79 | |
---|
| 80 | CMFSquidTool/threadinglocal.py - |
---|
| 81 | No cachefu-specific patching. It is basically a copy of 2.4.3 |
---|
| 82 | ``threading.local`` library. Didn't plone 2.5 basically require a |
---|
| 83 | python 2.4? In that case, this module might be entirely |
---|
| 84 | unnecessary. Ah, no. Plone 2.5 runs with zope 2.8 also, which runs |
---|
| 85 | on 2.3.5. |
---|
| 86 | |
---|
| 87 | |
---|
| 88 | More research: response headers and pagecache storing |
---|
| 89 | ----------------------------------------------------- |
---|
| 90 | |
---|
| 91 | Ehm, what about setting the response headers for something else than |
---|
| 92 | templates? And grabbing a nice fresh rendered content object for the |
---|
| 93 | pagecache? I expected that to need some patching, too. Haven't seen it |
---|
| 94 | till now, so more grepping is in order, especially for ``response``, |
---|
| 95 | as that is where headers get set. |
---|
| 96 | |
---|
| 97 | CacheSetup/cmf_utils.py - |
---|
| 98 | ``_setCacheHeaders()`` sets the cache headers according to cachefu's |
---|
| 99 | wishes. ``_checkConditionalGET()`` checks if a 304 should be send. |
---|
| 100 | |
---|
| 101 | This file doesn't do any patching, though. It gets used by |
---|
| 102 | ``patch_cmf.py``. |
---|
| 103 | |
---|
| 104 | PageCacheManager/PageCache.py - |
---|
| 105 | ``ZCache_get()`` and ``ZCache_set()`` retrieve or store pagecache |
---|
| 106 | items and set the headers (``X-pagecache``) accordingly. Hm, I |
---|
| 107 | almost forgot: items can be associated with cache managers. That's |
---|
| 108 | another angle to explore. Apparently cache managers automatically |
---|
| 109 | tie in at the start and finish of the process when associated with |
---|
| 110 | the content object that is being served. |
---|
| 111 | |
---|
| 112 | Looking at a wiki page where they discussed zope3's cache manager |
---|
| 113 | revealed that Tres Seaver had a strong preference for completely |
---|
| 114 | separating the caching of results (like in a cache manager) and the |
---|
| 115 | setting of headers. Those ought to be different products. Jim Fulton |
---|
| 116 | agreed. I also tend to agree :-) |
---|
| 117 | |
---|
| 118 | An almost logical result of this thinking is to look at the |
---|
| 119 | replacement cache managers offered by zope3 instead of at the |
---|
| 120 | existing zope2 cache managers as used nowadays by cachefu. |
---|
| 121 | |
---|
| 122 | Lovely systems made a viewcache_ for zope3 during the 2007 |
---|
| 123 | snowsprint. That one caches zope3 views. What is left to do is a |
---|
| 124 | cache for regular skin templates (pagecache could handle that for |
---|
| 125 | the time being, perhaps) and for python scripts. Well, python |
---|
| 126 | scripts are something separate: separate RAMcaches per script as per |
---|
| 127 | the instructions in the "speeding up zope" talk at the 2006 plone |
---|
| 128 | conf. |
---|
| 129 | |
---|
| 130 | .. _viewcache: http://www.lovelysystems.com/batlogg/2007/03/30/the-decathlon-of-computer-science/ |
---|
| 131 | |
---|
| 132 | PolicyHTTPCacheManager/PolicyHTTPCacheManager.py - |
---|
| 133 | |
---|
| 134 | This cache isn't meant to cache anything (sic). The goal is to be |
---|
| 135 | able to set headers just before the response gets returned. |
---|
| 136 | |
---|
| 137 | |
---|
| 138 | Conclusions/todo |
---|
| 139 | ---------------- |
---|
| 140 | |
---|
| 141 | * Figure out if the patching of OFS.Image/File is still necessary (to |
---|
| 142 | associate them with the PolicyHTTPCacheManager). Does zope3 offer |
---|
| 143 | something here? Does zope 2.8+ already do this? |
---|
| 144 | |
---|
| 145 | * Figure out if the IObjectmodified event covers all the cases that |
---|
| 146 | are now taken care of by the catalog patches. If so, replace those |
---|
| 147 | patches with an event subscriber that does the same thing. Real easy |
---|
| 148 | task, probably. |
---|
| 149 | |
---|
| 150 | * Does a change in ResourceRegistries also fires such an event? Then |
---|
| 151 | the resourceregistries patch can also go the way of the dodo. |
---|
| 152 | |
---|
| 153 | * Changing the permissions, the roles or the mapping between them: |
---|
| 154 | does this fire a specific event? Try to do away with the related |
---|
| 155 | patch, then. First, however, figure out where cachefu uses the |
---|
| 156 | related counter. |
---|
| 157 | |
---|
| 158 | * ``patch_cmf.py`` patches page templates so that 304 get handled and |
---|
| 159 | response headers get set: isn't that something that can be handled |
---|
| 160 | with a cache manaager? A cache manager seems to tie in at the |
---|
| 161 | correct locations. This might make the difference between the |
---|
| 162 | setting of headers and caching less clear. Hey, I'm even thinking |
---|
| 163 | about subclassing cachemanagers... I got an email by wiggy about an |
---|
| 164 | idea he and optilude had: a set of subscribers that can be iterated |
---|
| 165 | over instead of a one-choice-only event handler. |
---|
| 166 | |
---|
| 167 | * Macro caching. Remove it? I'm leaning in that direction. With |
---|
| 168 | lovely.viewcache, views can be cached. And views can also be |
---|
| 169 | viewlets: individual pieces of a page. At least for plone 3.0 |
---|
| 170 | that'll be a huge help. I do think that lovely.viewcache could use |
---|
| 171 | some cachefu UI love: it should be easy to choose certain views and |
---|
| 172 | identify the cache keys for them. For this, some of the macro code |
---|
| 173 | might come in real handy. |
---|
| 174 | |
---|
| 175 | Note: We've indeed removed it in trunk (i.e. for 3.0) on the PIKtipi |
---|
| 176 | Sprint (tomster, fschulze) |
---|
| 177 | |
---|
| 178 | * Figure out if ``CMFSquidTool/queue.py`` really duplicates what |
---|
| 179 | ``CacheSetup/patch.py`` does: catalog patching. Probably both can |
---|
| 180 | shed their mortal coil because an IObjectModified event handler is |
---|
| 181 | the way to go. Yeah, code removal! |
---|
| 182 | |
---|
| 183 | * Figure out if python 2.3.5 includes the treading.local library. If |
---|
| 184 | so, ``CMFSquidTool/threadinglocal.py`` can be cleaned out. |
---|
| 185 | |
---|
| 186 | * Investigate zope3's cache managers. What does lovely.viewcache do? |
---|
| 187 | Are viewcaches a handy way to tie into the publishing process or is |
---|
| 188 | it a dirty hack to use a cachemanager not to cache something but |
---|
| 189 | just to evaluate 304 matching and response header setting? |
---|
| 190 | |
---|
| 191 | * Can the PolicyHTTPCacheManager be replaced with something |
---|
| 192 | zope3-like? |
---|
| 193 | |
---|
| 194 | |
---|