mò )á¸Ec@sÒdZdkZdkZdkZdklZlZlZlZdk l Z dk l Z dk lZdkZdkZdkZd„Zdfd„ƒYZd fd „ƒYZd „Zd „Zd „Zd„ZedjoìdkZdadZdZdZdZdZ dZ!dZ"dZ#dZ%dZ&dZ'dZ(y,eiei)ddddgƒ\Z*Z+Wn"ei,j oZ-ee-ƒnXe.e+ƒdjoedƒnxJe*D]B\Z/Z0e/djoe1e0ƒaq²e/djoed7Zq²e/djoe1e0ƒZq²e/djo e0Z%q²e/djo e0Z&q²e/djoe1e0ƒZ!q²e/d jo dZq²e/d!joe1e0ƒZ q²e/d"joe1e0ƒZ"q²e/d#jo dZ#q²e/d$jo e0Z'q²e/d%jo e0Z(q²q²We+\Z2e'o(dk3Z3e3i4e'd&dd'dƒZ5n'e(odk'Z'e'i4ƒZ5ndZ5ee2e&e%e5ƒe'oe5i6ƒqÎe(oEdk7Z7e5i8e(ƒe7i9e(ƒZ:e:i;ƒi<d(ƒi=d)ƒqÎndS(*sÚTest an index with a Unix mailbox file. usage: python mailtest.py [options] options: -v -- verbose Index Generation -i mailbox -n NNN -- max number of messages to read from mailbox -t NNN -- commit a transaction every NNN messages (default: 1) -p NNN -- pack every NNN messages (default: 500), and at end -p 0 -- don't pack at all -x -- exclude the message text from the data.fs Queries -q query -b NNN -- return the NNN best matches (default: 10) -c NNN -- context; if -v, show the first NNN lines of results (default: 5) The script either indexes or queries depending on whether -q or -i is passed as an option. For -i mailbox, the script reads mail messages from the mailbox and indexes them. It indexes one message at a time, then commits the transaction. For -q query, it performs a query on an existing index. If both are specified, the index is performed first. You can also interact with the index after it is completed. Load the index from the database: import ZODB from ZODB.FileStorage import FileStorage fs = FileStorage( db = ZODB.DB(fs) index = cn.open().root()["index"] index.search("python AND unicode") N(sLexiconsCaseNormalizersSplittersStopWordRemover(s ZCTextIndex(sIOBTree(s QueryParsercCs|GHtGHtidƒdS(Ni(tmsgt__doc__tsystexit(R((tB/data/zmath/zope/lib/python/Products/ZCTextIndex/tests/mailtest.pytusage8stMessagecBstZdZd„ZRS(NicCs{|iddƒ}|iddƒ}|od||f}n d|}||iiƒ|_t i t |iƒ7_ dS(Ntsubjectttfroms%s (%s) s%s ( Rt getheaderRtauthortsummarytfptreadtselfttextRt total_bytestlen(RRR R R((Rt__init__As (t__name__t __module__RR(((RR=stExtracBstZRS(N(RR(((RRKscCs‘d} d}tiƒ} ttƒtƒtƒƒ}t ƒ}d|_ d|_ d|_ t ƒ}||_td||ƒ|d<}tptƒ|d<} nd} tiƒtit|dƒƒ}to dG|GHntp ti an|o(|i"t#||| |ƒ\} }}nt#||| |ƒ\} }}| | 7} ||7}tiƒt(o||t(djoktd jo d GHnti)ƒ}|i+tiƒƒti)ƒ} tod | |GHn|| |7}nto©tiƒ}Hd Gt.| d dƒGdGHdGt.|d dƒGdGHdGt/i0GHt/i0| d}d|GHdGti2| ƒGHdGti2|ƒGHdGt.|| d dƒGHndS(NitlexiconRsOkapi BM25 Ranktindext documentstrbtopenedispacking one last time...spack took %s secs Index timei<itminutess Pack times Index bytesisIndex rate %.2f KB/secsIndexing begansIndexing endedsWall clock minutes(3tidx_timet pack_timettimet start_timetLexicontSplittertCaseNormalizertStopWordRemoverRRtextrat lexicon_idtdoc_attrt index_typetcallert ZCTextIndextrttidxt EXCLUDE_TEXTtIOBTreetdocstNonet transactiontcommittmailboxt UnixMailboxtopentmboxfiletmboxtVERBOSEtNUMRtmaxinttprofilertruncallt indexmboxtdbtitimetptimetit PACK_INTERVALtclocktp0tpacktp1t finish_timetroundRRtratetctime(R+R6R>R;RR@R%RRIR?RFR/R RRDR,RAR)RGR7((RRNs^          (          c Cs£d} d}d} x| tjos|iƒ}|djoPn| d7} t|ƒ}t djo dG| GHnt i ƒ}|i| |ƒtp||| t cacheSizeRBRDRERF( R7R,R/R>RLRRMRRPRRFRDRA((RR=‡sB              cCs˜|d} |d} tiƒ}|djo| i|t ƒ\}} nKt odGH| i|t ƒntiƒ}|i| i|t ƒ\}} tiƒ|} dG|GHdGt|ƒGdG| Gd| dGHt| iƒi|ƒ}| ii|iƒƒ}x’|D]Š\}}d ||}d |||fGHtoW| |} | iid t ƒ}|d =d dGHdGHx|D] }|GHqtWd dGHqqWdS(NRRsWarming the cache...squery:s # results:tofs in %.2f msièf100.0s"docid %7d score %6d scaled %5.2f%%s iÿÿÿÿt-i<smessage:(#R+R,R/RRCtstartR;R0tqueryt query_strtBESTtresultst num_resultst WARM_CACHER<telapsedRt QueryParserRt parseQueryttreeRt query_weightttermstqwtdocidtscoretscaledR8RRtsplittCONTEXTtctxtl(R+RVR;RgRXRdRbRTRcRRYR/R[R,RaR^Rh((RRUªs:      #    cCs¦tii|ƒ}ti|dtƒ}|iƒ}|i ƒ}|dj ot ||||ƒn|dj ot|||ƒn|iƒ|iƒ|iƒdS(Nt cache_size(tZODBt FileStoragetfs_pathtftDBt CACHE_SIZER>R5tcntrootR+t mbox_pathR0RR;RVRUtclose(RlRrRVR;R+RpRmR>((RtmainÎs      t__main__iiôi'ii isvn:p:i:q:b:c:xt:wsprofile=s old-profile=s$exactly 1 filename argument requireds-ns-vs-ps-qs-is-bs-xs-ts-cs-ws --profiles --old-profilet lineeventst linetimingsRi(>RRjtZODB.FileStorageR1tProducts.ZCTextIndex.LexiconR!R#R"R$t Products.ZCTextIndex.ZCTextIndexR*tBTrees.IOBTreeR.t Products.ZCTextIndex.QueryParserR\RR3RRRRRR=RURtRtgetoptR9R8RBR-RoRORWRfRZR0RVRrtprofilet old_profiletargvtoptstargsterrorRRtotvtintRlthotshottProfileR;Rstpstatst dump_statstStatststatst strip_dirst sort_statst print_stats(*R…R‡RBR-R=RfRURWR*RRŒROR#RR}RtRVR~R"R.R‚R3RRR1RlRoRR\RZR‰R!R8RRrR„R;RjRRR$R((Rt?)s¢           9 # $