mò (á¸Ec@s¶dZdklZdkZdklZdkZdklZdk l Z dk Z dk l Z dklZdklZlZd „Zd fd „ƒYZd „Zd „ZedjoõdkZdkZdZdZdZdZdZeZ y#eiei!ddƒ\Z"Z#Wn/ei$j o Z%e%GHeGHei&dƒnXx´e"D]¬\Z'Z(e'djoed7Zne'djo e(Zne'djoe)e(ƒZne'djoe)e(ƒZne'djoe)e(ƒZne'djo eZ qOqOWe*e#ƒdjodGHeGHei&dƒne#dZ+eeƒZ,ei-e,ƒZ.e.i/ƒZ0e0i1ƒZ2ei3i4ei5ƒe+ƒZ+e+GHee.e2e+ƒe0i6ƒe,i6ƒndS(sÖIndex a collection of HTML files on the filesystem. usage: indexhtml.py [options] dir Will create an index of all files in dir or its subdirectories. options: -f data.fs -- the path to the filestorage datafile (s nested_scopesN(sclock(s FileStorage(sIOBTree(s ZCTextIndex(sHTMLWordSplitter(sLexiconsStopWordRemovercCs_dfd„ƒY}|ƒ}d|_d|_|ƒ}ttƒtƒƒ|_t d||ƒS(NtStructcBstZRS(N(t__name__t __module__(((tC/data/zmath/zope/lib/python/Products/ZCTextIndex/tests/indexhtml.pyRstreadtlexicon( Rtextratdoc_attrt lexicon_idtcallertLexicontHTMLWordSplittertStopWordRemoverRt ZCTextIndex(RRR ((Rt make_zc_indexs    t MySplittercBstZd„Zd„ZRS(NcCstƒ|_dS(N(R tselft _v_splitter(R((Rt__init__*scs4|ii|ƒ}‡d†}tdt||ƒƒS(Ncsˆi||ƒS(N(tstopdicttgettw(R(R(Rtlookup.s( RRt_splitttexttwordsRtfiltertNonetmap(RRRtargstkwargsRR((RRt__call__,s (RRRR(((RR)s cCsRdkl}dkl}dkl}||ƒƒ}tƒ|_|dd|ƒS(N(s TextIndex(sLexicon(s get_stopdictRR( t*Products.PluginIndexes.TextIndex.TextIndext TextIndext(Products.PluginIndexes.TextIndex.LexiconR tProducts.ZCTextIndex.StopDictt get_stopdicttlRt SplitterFunc(R$R R%R!((Rtmake_old_index2s     cCsžtƒtd<}tƒtd<} tiƒd}d}g}t i |ƒD]}|t ii||ƒqL~} d}tƒ}x±| D]©}t ii|ƒoA| g}t i |ƒD]}|t ii||ƒq·~7} q‡|idƒpq‡n|d7}tdj o|tjoPntod|G|GHnt|dƒ}|| |<|i||ƒ|iƒ|tdjoJtƒ} tiƒtƒ}||| 7}tod G|| G|GHqÇn|t!djoXtƒ}|i$ƒtƒ} || |7}|| |7}tod G| |G|GHq0q‡q‡Wtƒ} tiƒtƒ}}||} ||| 7}to'd G| GHd G| |GHd G| |GHndS(Ntindextfilesf0.0is.htmlis%5dtrbs commit tooks pack tooksTotal index times Non-pack times Non-ZODB time((tINDEXtrtR(tIOBTreetpathst transactiontcommitt zodb_timet pack_timet_[1]tostlistdirtdirtfiletpathtjoinR)tdocidtclocktt0tisdirtsubtendswithtLIMITRtVERBOSEtopentft index_objecttcloset TXN_INTERVALtz0tz1t PACK_INTERVALtp0tdbtpacktp1tt1t total_time(RKtrootR6R1R2R7R(R:R>R)R.RMRORGRHRJRCR<RNR3((Rtmain;s^ 9 A               t__main__isData.fsidiôis vf:t:p:n:Tis-vs-fs-ts-ps-ns-TsExpected on argument(7t__doc__t __future__t nested_scopesR4ttimeR;tZODBtZODB.FileStoraget FileStoragetBTrees.IOBTreeR-R/t Products.ZCTextIndex.ZCTextIndexR t!Products.ZCTextIndex.HTMLSplitterR tProducts.ZCTextIndex.LexiconR R RRR'RQRtsystgetoptRAtFSPATHRFRIRR@R+targvtoptsRterrortmsgtexittotvtinttlenR6tfstDBRKRBtcnRPR,R8R9tgetcwdRE(!R+RlRIR@RR'RR;RYRdR_R`RQR,RjR-RURFRRKR^RWR R R R/RARfR6RgR4R Rb((Rt? sr           3   #