1 | A cachefu with less monkeypatching |
---|
2 | ================================== |
---|
3 | |
---|
4 | Current patching in cachefu |
---|
5 | --------------------------- |
---|
6 | |
---|
7 | Currently, cachefu does a lot of monkeypatching. This is a list of the |
---|
8 | files in which monkeypatching takes place: |
---|
9 | |
---|
10 | CacheSetup/patch.py - |
---|
11 | OFS.Image/File are patched so that they're associated with the |
---|
12 | PolicyHTTPCacheManager to get control over old-style files and |
---|
13 | images. |
---|
14 | |
---|
15 | A ``.modified()`` method is patched into old-style files and images |
---|
16 | and into filesystem files and images to support asking them about a |
---|
17 | last modification date. |
---|
18 | |
---|
19 | The catalog is patched: ``moveObjectsByDelta()``, |
---|
20 | ``uncatalog_object()`` and ``catalog_object()``. All now call a |
---|
21 | method that purges squid (except for uncatalog_object) so that the |
---|
22 | object in squid gets purged whenever it changes. "Changes" in this |
---|
23 | case is defined as "when the catalog changes". It was a good way to |
---|
24 | detect it for older plones, but at the moment not everything is |
---|
25 | detected this way. IObjectModified event to the resque? |
---|
26 | |
---|
27 | The squid purge method also increments a counter (also for |
---|
28 | uncatalog_object) inside the cache tool: this is used as a sort |
---|
29 | last-modified-date for the catalog. This counter is used quite a lot |
---|
30 | by cachefu to keep everything updated, especially for logged in |
---|
31 | users. |
---|
32 | |
---|
33 | ResourceRegistries' ``cookResources()`` is patched so that |
---|
34 | abovementioned catalog counter gets incremented everything something |
---|
35 | changes inside ResourceRegistries. |
---|
36 | |
---|
37 | ``manage_changePermissions()``, ``manage_permission()``, |
---|
38 | ``manage_acquiredPermissions()`` and ``manage_role()`` are patched |
---|
39 | to increment a counter every time the relationship between |
---|
40 | permissions and roles change. This counter can then be used to |
---|
41 | determine freshness. |
---|
42 | |
---|
43 | CacheSetup/patch_cmf.py - |
---|
44 | ``FSPageTemplate`` gets patched so that the filesystem (=skin |
---|
45 | directory) template doesn't get rendered if a "304 not modified" can |
---|
46 | be returned. Rendering the template is expensive and it isn't needed |
---|
47 | if a 304 can be returned. Secondly, cache headers are set after the |
---|
48 | page template has been rendered. |
---|
49 | |
---|
50 | The pagetemplate rendering mechanism, ``TALInterpreter``, is patched |
---|
51 | likewise. 304 interruption and cache heading setting. |
---|
52 | |
---|
53 | CacheSetup/patch_interpreter.py - |
---|
54 | This is a patch to enable macro caching. It looks like it only works |
---|
55 | with plone 2.5 (not 3.0) at the moment because of the import of the |
---|
56 | TAL.TALInterpreter, which isn't available anymore in plone 3.0's |
---|
57 | zope version. |
---|
58 | |
---|
59 | Many of the people at the sprint didn't seem to like macro caching, |
---|
60 | so it might be best to just ignore it for the time being. In that |
---|
61 | case the patch can stay, but will only be useful for 2.5. |
---|
62 | |
---|
63 | Note: We've indeed removed it in trunk (i.e. for 3.0) on the PIKtipi |
---|
64 | Sprint (tomster, fschulze) |
---|
65 | |
---|
66 | CacheSetup/patch_utils.py - |
---|
67 | Actually, this is just a utility class that provides some handy |
---|
68 | methods for patching files. |
---|
69 | |
---|
70 | CMFSquidTool/patch.py - |
---|
71 | Just a utility class for patching. What's the possibility for |
---|
72 | merging it with above one? |
---|
73 | |
---|
74 | CMFSquidTool/queue.py - |
---|
75 | ``reindexObject()`` and ``unindexObject()`` of CMFCatalogAware and |
---|
76 | CatalogMultiplex are patched to put objects in the squid purge |
---|
77 | queue. Didn't ``CacheSetup/patch.py`` already to that? Might be a |
---|
78 | chance for some serious code cleanup here. |
---|
79 | |
---|
80 | CMFSquidTool/threadinglocal.py - |
---|
81 | No cachefu-specific patching. It is basically a copy of 2.4.3 |
---|
82 | ``threading.local`` library. Didn't plone 2.5 basically require a |
---|
83 | python 2.4? In that case, this module might be entirely |
---|
84 | unnecessary. Ah, no. Plone 2.5 runs with zope 2.8 also, which runs |
---|
85 | on 2.3.5. |
---|
86 | |
---|
87 | |
---|
88 | More research: response headers and pagecache storing |
---|
89 | ----------------------------------------------------- |
---|
90 | |
---|
91 | Ehm, what about setting the response headers for something else than |
---|
92 | templates? And grabbing a nice fresh rendered content object for the |
---|
93 | pagecache? I expected that to need some patching, too. Haven't seen it |
---|
94 | till now, so more grepping is in order, especially for ``response``, |
---|
95 | as that is where headers get set. |
---|
96 | |
---|
97 | CacheSetup/cmf_utils.py - |
---|
98 | ``_setCacheHeaders()`` sets the cache headers according to cachefu's |
---|
99 | wishes. ``_checkConditionalGET()`` checks if a 304 should be send. |
---|
100 | |
---|
101 | This file doesn't do any patching, though. It gets used by |
---|
102 | ``patch_cmf.py``. |
---|
103 | |
---|
104 | PageCacheManager/PageCache.py - |
---|
105 | ``ZCache_get()`` and ``ZCache_set()`` retrieve or store pagecache |
---|
106 | items and set the headers (``X-pagecache``) accordingly. Hm, I |
---|
107 | almost forgot: items can be associated with cache managers. That's |
---|
108 | another angle to explore. Apparently cache managers automatically |
---|
109 | tie in at the start and finish of the process when associated with |
---|
110 | the content object that is being served. |
---|
111 | |
---|
112 | Looking at a wiki page where they discussed zope3's cache manager |
---|
113 | revealed that Tres Seaver had a strong preference for completely |
---|
114 | separating the caching of results (like in a cache manager) and the |
---|
115 | setting of headers. Those ought to be different products. Jim Fulton |
---|
116 | agreed. I also tend to agree :-) |
---|
117 | |
---|
118 | An almost logical result of this thinking is to look at the |
---|
119 | replacement cache managers offered by zope3 instead of at the |
---|
120 | existing zope2 cache managers as used nowadays by cachefu. |
---|
121 | |
---|
122 | Lovely systems made a viewcache_ for zope3 during the 2007 |
---|
123 | snowsprint. That one caches zope3 views. What is left to do is a |
---|
124 | cache for regular skin templates (pagecache could handle that for |
---|
125 | the time being, perhaps) and for python scripts. Well, python |
---|
126 | scripts are something separate: separate RAMcaches per script as per |
---|
127 | the instructions in the "speeding up zope" talk at the 2006 plone |
---|
128 | conf. |
---|
129 | |
---|
130 | .. _viewcache: http://www.lovelysystems.com/batlogg/2007/03/30/the-decathlon-of-computer-science/ |
---|
131 | |
---|
132 | PolicyHTTPCacheManager/PolicyHTTPCacheManager.py - |
---|
133 | |
---|
134 | This cache isn't meant to cache anything (sic). The goal is to be |
---|
135 | able to set headers just before the response gets returned. |
---|
136 | |
---|
137 | |
---|
138 | Conclusions/todo |
---|
139 | ---------------- |
---|
140 | |
---|
141 | * Figure out if the patching of OFS.Image/File is still necessary (to |
---|
142 | associate them with the PolicyHTTPCacheManager). Does zope3 offer |
---|
143 | something here? Does zope 2.8+ already do this? |
---|
144 | |
---|
145 | * Figure out if the IObjectmodified event covers all the cases that |
---|
146 | are now taken care of by the catalog patches. If so, replace those |
---|
147 | patches with an event subscriber that does the same thing. Real easy |
---|
148 | task, probably. |
---|
149 | |
---|
150 | * Does a change in ResourceRegistries also fires such an event? Then |
---|
151 | the resourceregistries patch can also go the way of the dodo. |
---|
152 | |
---|
153 | * Changing the permissions, the roles or the mapping between them: |
---|
154 | does this fire a specific event? Try to do away with the related |
---|
155 | patch, then. First, however, figure out where cachefu uses the |
---|
156 | related counter. |
---|
157 | |
---|
158 | * ``patch_cmf.py`` patches page templates so that 304 get handled and |
---|
159 | response headers get set: isn't that something that can be handled |
---|
160 | with a cache manaager? A cache manager seems to tie in at the |
---|
161 | correct locations. This might make the difference between the |
---|
162 | setting of headers and caching less clear. Hey, I'm even thinking |
---|
163 | about subclassing cachemanagers... I got an email by wiggy about an |
---|
164 | idea he and optilude had: a set of subscribers that can be iterated |
---|
165 | over instead of a one-choice-only event handler. |
---|
166 | |
---|
167 | * Macro caching. Remove it? I'm leaning in that direction. With |
---|
168 | lovely.viewcache, views can be cached. And views can also be |
---|
169 | viewlets: individual pieces of a page. At least for plone 3.0 |
---|
170 | that'll be a huge help. I do think that lovely.viewcache could use |
---|
171 | some cachefu UI love: it should be easy to choose certain views and |
---|
172 | identify the cache keys for them. For this, some of the macro code |
---|
173 | might come in real handy. |
---|
174 | |
---|
175 | Note: We've indeed removed it in trunk (i.e. for 3.0) on the PIKtipi |
---|
176 | Sprint (tomster, fschulze) |
---|
177 | |
---|
178 | * Figure out if ``CMFSquidTool/queue.py`` really duplicates what |
---|
179 | ``CacheSetup/patch.py`` does: catalog patching. Probably both can |
---|
180 | shed their mortal coil because an IObjectModified event handler is |
---|
181 | the way to go. Yeah, code removal! |
---|
182 | |
---|
183 | * Figure out if python 2.3.5 includes the treading.local library. If |
---|
184 | so, ``CMFSquidTool/threadinglocal.py`` can be cleaned out. |
---|
185 | |
---|
186 | * Investigate zope3's cache managers. What does lovely.viewcache do? |
---|
187 | Are viewcaches a handy way to tie into the publishing process or is |
---|
188 | it a dirty hack to use a cachemanager not to cache something but |
---|
189 | just to evaluate 304 matching and response header setting? |
---|
190 | |
---|
191 | * Can the PolicyHTTPCacheManager be replaced with something |
---|
192 | zope3-like? |
---|
193 | |
---|
194 | |
---|