m mEc@sdZdkZdkZdkZdkZdkZdkZdkZdkZdk Z dk Z dk Z dk Z dk Z dkZdk lZlZlZdklZy dkZWnej odkZnXdkadklZdklZlZlZlZl Z dk!l"Z"l#Z#dk$l%Z%dk&l'Z'l(Z(ei)d i*Z*d Z+d Z,d efd YZ-defdYZ.de/fdYZ0dfdYZ1y dk2Z2Wnej onXde1e2i2fdYZ3de1ei4fdYZ5dZ6defdYZ7defdYZ8defdYZ9y dk:Z:Wnej on1Xde ifd YZ;d!efd"YZ<d#efd$YZ=d%efd&YZ>d'efd(YZ?d)efd*YZ@d+efd,YZAd-efd.YZBd/eBfd0YZCeDe d1od2eBfd3YZEnd4fd5YZFeFiGZGdaeiIZJdd6ZKdddd7ZLd8ZMdS(9sIntegration with Python standard library module urllib2. Also includes a redirection bugfix, support for parsing HTML HEAD blocks for the META HTTP-EQUIV tag contents, and following Refresh header redirects. Copyright 2002-2006 John J Lee This code is free software; you can redistribute it and/or modify it under the terms of the BSD or ZPL 2.1 licenses (see the file COPYING.txt included with the distribution). N(sURLErrors HTTPErrors BaseHandler(sStringIO(sRequest(s isstringlikes startswiths getheadersscloseable_responsesresponse_seek_wrapper(sunescapesunescape_charref(sis_html(s CookieJars request_hostsmechanize.cookiesislatin-1tHTTPRedirectHandlercBs@tZdZdZdZdZeZZZeZ dZ RS(Nii cCss|d jp|djo7|i o)t|d|id|idtSnt|i ||||d S( snReturn a Request or None in response to a redirect. This is called by the http_error_30x methods when a redirection response is received. If a redirection should take place, return a new Request to allow http_error_30x to perform the redirect; otherwise, return None to indicate that an HTTPError should be raised. i-i.i/trefreshi3theaderstorigin_req_hostt unverifiableN(i-i.i/srefresh( tcodetreqthas_datatRequesttnewurlRtget_origin_req_hosttTruet HTTPErrort get_full_urltmsgtfp(tselfR RRRRR((t9/data/zmath/zope/lib/python/mechanize/_urllib2_support.pytredirect_requestBs (  c Csn|idot|dd}n,|idot|dd}ndSti|i|}|i ||||||}|djodSnt|doo|i}|_|i|d|ijpt||ijo)t|i||i|||q0nh}|_|_|i|dd||<|i|i|ii|S(Ntlocationiturit redirect_dicti(Rthas_keyt getheadersR turlparseturljoinRR RRRRRtnewtNonethasattrRtvisitedtgett max_repeatstlentmax_redirectionsR tinf_msgtreadtclosetparenttopen( RRRRRRR RR((Rthttp_error_302Zs& 2  soThe HTTP server returned a redirect error that would lead to an infinite loop. The last 30x error message was: ( t__name__t __module__RR!RR'thttp_error_301thttp_error_303thttp_error_307thttp_error_refreshR"(((RR)s  %tHTTPRequestUpgradeProcessorcBstZdZdZeZRS(NicCst|dpqt|i|i|i}y|i|_Wntj onXy|i |_ Wntj onX|}n|S(Ntadd_unredirected_header( RtrequestRt_Request__originaltdataRt newrequestRtAttributeErrorR(RR0R3((Rt http_requests  (R(R)t handler_orderR5t https_request(((RR.s tEndOfHeadErrorcBstZRS(N(R(R)(((RR8stAbstractHeadParserc BsntZdZeiZeZd Zd Z d Z d Z dZ dZ dZdZdZRS(NthtmltheadttitletbasetscripttstyletmetatlinktobjectcCs g|_dS(N(Rt http_equiv(R((Rt__init__scCsd}}xT|D]L\}}|djo|i|}q|djo|i|}qqW|dj o|ii||fndS(Ns http-equivtcontent( RRCREtattrstkeytvalueRtunescape_attr_if_requiredtappend(RRFRCRHRERG((Rt start_metas     cCs tdS(N(R8(R((Rtend_headscCs'|itd||i|idS(Ns&%s;(Rt handle_datatunescapetnamet _entitydefst _encoding(RRO((Rthandle_entityrefscCs|it||idS(N(RRMtunescape_charrefRORQ(RRO((Rthandle_charrefscCst||i|iS(N(RNRORRPRQ(RRO((Rt unescape_attrscCs:h}x-|iD]\}}|i|||yt|d|}Wntj oqX||n X||dS(Ntstart_tdo_(ttagRR]R8tgetattrtmethodR4RF(RReRFRg((Rthandle_starttags cCsU||ijo tnyt|d|}Wntj on X|dS(Ntend_(ReRR]R8RfRgR4(RReRg((Rt handle_endtags cCs |i|S(N(RRURO(RRO((RRNscCs|S(N(RO(RRO((RRIs(R(R)RDRhRjRNRI(((RRas    t HeadParsercBs>tZdZdZdZdZdZdZRS(NcCstptdS(N(tFalsetAssertionError(R((Rt _not_calledscCs!tii|ti|dS(N(tsgmllibt SGMLParserRDRR9(R((RRDscCs<||ijo tn|djo||ndS(NR@(ReRR]R8RgRF(RReRgRF((RRhs  cCs|i||i|dS(N(RRhReRnRF(RReRF((Rtunknown_starttag scCs(||ijo |n tdS(N(ReRR]RgR8(RReRg((RRj s cCs |i|S(N(RRURO(RRO((RRIs(R(R)RnRDRhRqRjRI(((RRks      cCs^xT|it}y|i|Wntj oPnXt|tjoPqqW|iS(s"Return a list of key, value pairs.iN( tfileobjR#tCHUNKR2tparsertfeedR8R RC(RrRtR2((Rt parse_heads tHTTPEquivProcessorcBs2tZdZdZeedZdZeZRS(s7Append META HTTP-EQUIV headers to regular HTTP headers.i,cCs||_||_dS(N(thead_parser_classRti_want_broken_xhtml_supportt _allow_xhtml(RRxRy((RRD(s c Cst|dpt|}n|i}|i}t|id}t |||i ouy.zt ||i }Wd|idXWntitifj oqXx"|D]\}}|||Perform HTTP Refresh redirections. Note that if a non-200 HTTP code has occurred (for example, a 30x redirect), this processor will do nothing. By default, only zero-time Refresh headers are redirected. Use the max_time attribute / constructor argument to allow Refresh with longer pauses. Use the honor_time attribute / constructor argument to control whether the requested pause is honoured (with a time.sleep()) or skipped in favour of immediate redirection. Public attributes: max_time: see above honor_time: see above iicCs||_||_dS(N(tmax_timeRt honor_time(RRR((RRDs c Cs|i|i|i}} }|djod|idoTt|dd} ti | d}|djot | | | |d} }ti |d} | djo|| || d}}n|iidjotd | |Sqnt | |i} }|idjp| |ijoW| d jo|ioti| n||d <|iid ||d| |}qn|S( NiRit;iit=Rsbad Refresh header: %rf0.001Rthttp(R|RRR~thdrsRRRtstringtfindtiitfloattpauset newurl_spectjjRGR tstriptlowertdebugRRRRRttimetsleepR%terrorR0( RR0R|RGRRRRR RRRR((RRs(      (R(R)RR6R RDRR(((RRs  tHTTPErrorProcessorcBs#tZdZdZdZeZRS(sProcess HTTP error responses. The purpose of this handler is to to allow other response processors a look-in by removing the call to parent.error() from AbstractHTTPHandler. For non-200 error codes, this just passes the job on to the Handler._error_ methods, via the OpenerDirector.error method. Eventually, urllib2.HTTPDefaultErrorHandler will raise an HTTPError if no other handler handles the error. icCsV|i|i|i}}}|djo%|iid|||||}n|S(NiR( R|RRR~RRR%RR0(RR0R|RRR((RR s  %(R(R)RR6RR(((RRs  tAbstractHTTPHandlercBs/tZddZdZdZdZRS(NicCs ||_dS(N(t debuglevelRt _debuglevel(RR((RRDscCs ||_dS(N(tlevelRR(RR((Rtset_http_debuglevelsc Cs|i}|ptdn|io4|i}|idp|iddqdnt i |i \}}t i|\} }|idp|id| p|nxM|iiD]?\}}ti|}|i|p|i||qqW|S(Ns no host givens Content-types!application/x-www-form-urlencodedtHost(R0RRtURLErrorRtget_dataR2RR/turllibt splittypet get_selectorRtselt splithosttsel_hosttsel_pathRR%t addheadersRORHRt capitalize( RR0RRORRHRRR2R((Rt do_request_!s$     c Cs|i}|ptdn||}|i|i|i i }|i |i d|d|D]6}t |t ijo |}n|i|q W|S(sFCreate an opener object from a list of handlers and processors. The opener will use several default handlers and processors, including support for HTTP and FTP. If any of the handlers passed as arguments are subclasses of the default handlers, the default handlers will not be used. RN(RRtopenertlisttdefault_classesRRRJRtskipthandlerstcheckttypettypest ClassTypet issubclasst InstanceTypet isinstancetremovet add_handlerR(RRRRRRRR((Rt build_openers8   (R(R)RRt ProxyHandlertUnknownHandlerRtHTTPDefaultErrorHandlerRt FTPHandlert FileHandlerR.RRRRtreplacement_handlerst_openertOpenerDirectorRDR(((RRxs 3cCsWtdjo:tiztdjo tanWdtiXnti||S(N( R Rt urlopen_locktacquireRtreleaseR&RR2(RR2((Rturlopens   cCs]tdjo:tiztdjo tanWdtiXnti||||S(N( R RR R RRtretrieveRtfilenamet reporthookR2(RRRR2((Rt urlretrieves   cCs |adS(N(RR (R((Rtinstall_openers(NRRRttempfileR^tretloggingRRRRRRRRoRR t BaseHandlert cStringIORt threadingt _threadingt ImportErrortdummy_threadingR t_requestRt_utilt isstringliket startswithRRR}t_htmlRNRSt _headersutilRt _clientcookieRt request_hostRRRsR`RR.t ExceptionR8R9RbRaRpRkRvRwRRRRRRRRRRRRRRRRRtLockR RRR(=RRRRRRRbRRRRkRR9RoRRR}RsR RRRRRRRR8RR!R.RRRR^RRRRaRRRNR RRRRRRR RRvR`RRRRRRRwRSR%((Rt? sh~    % ^3 ' " '5R: