m MEc@sdZdkZdkZdkZdkZdkZdkZdkZdkZdk l Z dk l Z dk lZdklZlZlZdklZdklZdklZd klZd klZeid Zeei Z!ei"e!e#d Z$yd k%l&Z&Wne'j o dZ&nXdZ(e)dZ*dfdYZ+e+Z,ddZ-de.fdYZ/dZ0dS(sThe ClientStorage class and the exceptions that it may raise. Public contents of this module: ClientStorage -- the main class, implementing the Storage API N(s ServerStub(s ClientCache(sTransactionBuffer(sClientStorageErrorsClientDisconnecteds AuthError(s get_module(sConnectionManager(s POSException(sBLATHER(s TimeStampsZEO.ClientStoragecCs*d||f}ti||d|dS(Ns(%s) %stexc_info(tsubsystmsgtmessagetloggertlogtlevelR(RRRRR((t0/data/zmath/zope/lib/python/ZEO/ClientStorage.pytlog2,s(sResolvedSerialtrscCstt|S(N(tstrt TimeStampttid(R ((Rttid2time5scCsTti}tti|d |df}|dj o|i|}n|S(sInternal helper to return a unique TimeStamp instance. If the optional argument is not None, it must be a TimeStamp; the return value is then guaranteed to be at least 1 microsecond later the argument. ii<N(ttimettR tgmtimetprev_tstNonet laterThan(RR((Rt get_timestamp8s  $ tDisconnectedServerStubcBstZdZdZRS(s Internal helper class used as a faux RPC stub when disconnected. This raises ClientDisconnected on all attribute accesses. This is a singleton class -- there should be only one instance, the global disconnected_stub, os it can be tested by identity. cCs tdS(N(tClientDisconnected(tselftattr((Rt __getattr__Ns(t__name__t __module__t__doc__R(((RREs iit ClientStoragecBstZdZeZeZeZe i Z dde de de dde e e dddde dZe dZd Zd Zd Zd Zd ZdZdZdZdZdZdZdZdZdZdZdZdZ dZ!dZ"dZ#dZ$dZ%dZ&d d!Z'e d"Z(d#Z)d$Z*d%Z+d&Z,d'Z-d(Z.d)Z/e e d dd*Z0d+Z1d,Z2d-Z3e d.d/Z4d0Z5d1Z6d2Z7e d3Z8d4Z9d5Z:dd6e d7Z;dd6e d8Z<d9Z=e d:Z>d;Z?d<Z@d=ZAd>ZBd?ZCd@ZDdAZEeAZFeCZGeEZHRS(BsA Storage class that is a network client to a remote storage. This is a faithful implementation of the Storage API. This class is thread-safe; transactions are serialized in tpc_begin(). t1itiii,cCs td|iiti| odpd|odpd|f|otdn| dj oE| dj o$| | jotddt iqtd | } n| djo d } n||_t|_d|_d|_ti|_| |_d |_||_||_||_||_||_!d |_"d|_#d|_$d|_%hd d <d d <dd<dd <dd <dd <|_&|i'|_(d|_)d|_*g|_+h|_,|p t.||_ti/|_0d|_1ti2|_3g|_4ti2|_5d|_6d|_7ti2|_8|dj o6|p ti;}ti=i>|d||f}nd}|i@|d ||_B|iBiC|iD||d|d| |_G| o|iH| n"|iGiJp|iGiKndS(s_ ClientStorage constructor. This is typically invoked from a custom_zodb.py file. All arguments except addr should be keyword arguments. Arguments: addr -- The server address(es). This is either a list of addresses or a single address. Each address can be a (hostname, port) tuple to signify a TCP/IP connection or a pathname string to signify a Unix domain socket connection. A hostname may be a DNS name or a dotted IP address. Required. storage -- The storage name, defaulting to '1'. The name must match one of the storage names supported by the server(s) specified by the addr argument. The storage name is displayed in the Zope control panel. cache_size -- The disk cache size, defaulting to 20 megabytes. This is passed to the ClientCache constructor. name -- The storage name, defaulting to ''. If this is false, str(addr) is used as the storage name. client -- A name used to construct persistent cache filenames. Defaults to None, in which case the cache is not persistent. See ClientCache for more info. debug -- Ignored. This is present only for backwards compatibility with ZEO 1. var -- When client is not None, this specifies the directory where the persistent cache files are created. It defaults to None, in whichcase the current directory is used. min_disconnect_poll -- The minimum delay in seconds between attempts to connect to the server, in seconds. Defaults to 5 seconds. max_disconnect_poll -- The maximum delay in seconds between attempts to connect to the server, in seconds. Defaults to 300 seconds. wait_for_server_on_startup -- A backwards compatible alias for the wait argument. wait -- A flag indicating whether to wait until a connection with a server is made, defaulting to true. wait_timeout -- Maximum time to wait for a connection before giving up. Only meaningful if wait is True. read_only -- A flag indicating whether this should be a read-only storage, defaulting to false (i.e. writing is allowed by default). read_only_fallback -- A flag indicating whether a read-only remote storage should be acceptable as a fallback when no writable storages are available. Defaults to false. At most one of read_only and read_only_fallback should be true. username -- string with username to be used when authenticating. These only need to be provided if you are connecting to an authenticated server storage. password -- string with plaintext password to be used when authenticated. Note that the authentication protocol is defined by the server and is detected by the ClientStorage upon connecting (see testConnection() and doAuth() for details). s)%s (pid=%d) created %s/%s for storage: %rtROtRWtfallbacktnormals1ClientStorage(): debug argument is no longer usedsZClientStorage(): conflicting values for wait and wait_for_server_on_startup; wait prevailsRsRClientStorage(): wait_for_server_on_startup is deprecated; please use wait insteadiitlengthtsizetnames ZEO Clientt supportsUndotsupportsVersionstsupportsTransactionalUndos %s-%s.zecttminttmaxN(LRRt __class__Rtostgetpidt read_onlytread_only_fallbacktstoragetdebugtwait_for_server_on_startupRtwaittloggingtWARNINGtaddrt_addrtdisconnected_stubt_servert _connectiont_pending_servert threadingtEventt_readyt _is_read_onlyt_conn_is_read_onlyt_storaget_read_only_fallbacktusernamet _usernametpasswordt _passwordtrealmt_realmt_midtxn_disconnectt _server_addrt_tfilet_picklert_infotTransactionBufferClasst_tbuft_dbt_ltidt_serialst_serialdR&R t Conditiont _tpc_condt _transactiontLockt _oid_lockt_oidst _load_lockt _load_oidt _load_statust_locktclienttvartgetcwdtdirtpathtjoint cache_pathtClientCacheClasst cache_sizet_cachetopentConnectionManagerClasstmin_disconnect_polltmax_disconnect_pollt_rpc_mgrt_waitt wait_timeouttattempt_connecttconnect(RR7R1RgR&R_R2R`RkRlR3R4RoR/R0RDRFRHReRb((Rt__init__gspPE                    ?         #  cCs|dj o(ti|}td|dtnd}|iidd|i i pt xf|i i d|i ioPn|o+ti|jotddtiPntdqhWdS(NsSetting deadline to %fRtsynciis Timed out waiting for connections(Waiting for cache verification to finish(ttimeoutRRtdeadlineRtBLATHERRRmRqR;tis_asynctAssertionErrorR?R4tisSetR5R6(RRtRu((RRnIs   cCse|ii|idj o|iid|_n|idj o|iid|_ndS(s@Storage API: finalize the storage, releasing external resources.N(RRPtcloseRhRRm(R((RRzas    cCs ||_dS(sStorage API: register a database for invalidation messages. This is called by ZODB.DB (and by some tests). The storage isn't really ready to use until after this call. N(tdbRRQ(RR{tlimit((Rt registerDBkscCs |iiS(s>Return whether the storage is currently connected to a server.N(RR?Ry(R((Rt is_connectedtscCsdS(N((R((RRszscCs|io|iptdnt|}|p+td|ii |fdt i dSn|\}}}|p3td|ii |fdt i tdn||}|i|i|i|iS(Nsempty username or passwords %s: no such an auth protocol: %sRs7%s: %s isn't a valid protocol, must have a Client classsinvalid protocol(RRERGt AuthErrort get_moduletprotocoltmoduleRR,RR5R6t storage_classR_tdb_clasststubtctstartRI(RRRRRRR_R((RtdoAuth~s    cCstd|d|_|i|}|i}td||oN|i||}|otd|i |qtdt dny$|i t |i|idSWnZtij oK|ipntd|i t |iddd|_dSnXd S( s Internal: test the given connection. This returns: 1 if the connection is an optimal match, 0 if it is a suboptimal but acceptable match. It can also raise DisconnectedError or ReadOnlyError. This is called by ZEO.zrpc.ConnectionManager to decide which connection to use in case there are multiple, and some are read-only and others are read-write. This works by calling register() on the server. In read-only mode, register() is called with the read_only flag set. In writable mode and in read-only fallback mode, register() is called with the read_only flag cleared. In read-only fallback mode only, if the register() call raises ReadOnlyError, it is retried with the read-only flag set, and if this succeeds, this is deemed a suboptimal match. In all other cases, a succeeding register() call is deemed an optimal match, and any exception raised by register() is passed through. sTesting connection %ris!Server authentication protocol %rs Client authentication successfulsAuthentication failedis0Got ReadOnlyError; trying again with read_only=1R/N(RtconnRRAtStorageServerStubClassRtgetAuthProtocoltauthRtskeyt setSessionKeyRtregisterR RBR@t POSExceptiont ReadOnlyErrorRC(RRRRR((RttestConnections.       cCs |idjodSn|idj o|iin|idj o d}nd}|i|i |idj o|ii n||_|ot d|i nt d|i |i |}g|_|i||ii|i|ipt|idS(sInternal: start using the given connection. This is called by ConnectionManager after it has decided which connection should be used. NiisReconnected to storage: %ssConnected to storage: %s(RRhRRQtinvalidateCacheR;t reconnecttset_server_addrRtget_addrRzRRKRRRZt verify_cacheRNtupdatetget_infoRwRxt_handle_extensions(RRRR((RtnotifyConnecteds*    cCsQxJ|iiD]6}t||p t|||ii|qqWdS(N(RtgetExtensionMethodstkeysR&thasattrtsetattrR:textensionMethod(RR&((RRscCst|tio ||_nt|tipt|d}yt i |\}}}Wn8t ij o)}td||fdt|}nXt||df|_dS(NisError resolving host: %s (%s)Ri(t isinstanceR7ttypest StringTypeRRKt TupleTypeRxthosttsockett gethostbyaddrt canonicaltaliasestaddrsterrorterrRRvR (RR7RRRRR((RRs    cCs2|idjo tnd|i|ifSdS(Ns%s:%s(RRKRRRB(R((RtsortKeys cCs|ii}|d j o|i}||jo(td||_ |i i dSntd|t |ftd||o t |f|i |}|d j oCtdt|d|i|||_ |i i dSqntdtid d |_ti|id|_d|i_x3|iiD]"\}}}|i|||qNW||_|id S( sInternal routine called to verify the cache. The return value (indicating which path we took) is used by the test suite. s5No verification necessary (last_inval_tid up-to-date)sno verificationslast inval tid: %r %s slast transaction: %r %ssRecovering %d invalidationsisquick verificationsVerifying cachetsuffixs.invsfull verificationN(RRht getLastTidtlast_inval_tidRtservertlastTransactiontltidRR:R?tsetR tgetInvalidationstpairtlentinvalidateTransactionttempfilet TemporaryFileRLtcPickletPicklerRMtfasttcontentstoidR tversiontverifyR<t endZeoVerify(RRRRRRR R((RRs8      !         cCsCtdt|id|_|iit|_ d|_ dS(sInternal: notify that the server connection was terminated. This is called by ConnectionManager when the connection is closed or when certain problems with the connection occur. sDisconnected from storage: %siN( RtreprRRKRR;R?tclearR9R:RJ(R((RtnotifyDisconnectedNs    cCs |idS(sReturn the size of the storage.R$N(RRN(R((Rt__len__ZscCs%d|i|iodpdfS(sWStorage API: return the storage name as a string. The return value consists of two parts: the name as determined by the name and addr argments to the ClientStorage constructor, and the string 'connected' or 'disconnected' in parentheses indicating whether the storage is (currently) connected. s%s (%s)t connectedt disconnectedN(RRR~(R((RtgetName_scCs |idS(s;Storage API: an approximate size of the database, in bytes.R%N(RRN(R((RtgetSizelscCs|iidhS(sgetExtensionMethods This returns a dictionary whose keys are names of extra methods provided by this storage. Storage proxies (such as ZEO) should call this method to determine the extra methods that they need to proxy in addition to the standard storage methods. Dictionary values should be None; this will be a handy place for extra marshalling information, should we need it textensionMethodsN(RRNtget(R((RRps cCs |idS(s,Storage API: return whether we support undo.R'N(RRN(R((RR'|scCs |idS(s0Storage API: return whether we support versions.R(N(RRN(R((RR(scCs |idS(s:Storage API: return whether we support transactional undo.R)N(RRN(R((RR)scCs|iodSn|iSdS(s5Storage API: return whether we are in read-only mode.iN(RR@RA(R((Rt isReadOnlys cCsG|iotin|i|j oti|i|ndS(s;Internal helper to check a transaction argument for sanity.N(RR@RRRWttranstStorageTransactionError(RR((Rt _check_transs   cCs\|i||ii|t|\}}x!|D]}|i i |dq5W||fS(s9Storage API: clear any changes made by the given version.RN( RRttxnR:t abortVersionRtidR toidsRRPt invalidate(RRRRRR ((RRs ! cCs|i||ii||t|\}}|o(xI|D]}|i i ||q?Wn%x!|D]}|i i |dqgW||fS(s:Storage API: commit the source version in the destination.RN( RRRR:t commitVersiontsourcet destinationRR RRRPR(RRRRRRR ((RRs $icCs|ii|||S(sStorage API: return a sequence of HistoryEntry objects. This does not support the optional filter argument defined by the Storage API. N(RR:thistoryRRR$(RRRR$((RRscCs|ii|S(shStorage API: get the mext database record. This is part of the conversion-support API. N(RR:trecord_iternexttnext(RR((RRscCs|ii|S(s2Storage API: return current serial number for oid.N(RR:t getSerialR(RR((RRscCs|ii||S(s5Storage API: load a historical revision of an object.N(RR:t loadSerialRtserial(RRR((RRscCs|i||d S(sStorage API: return the data for a given object. This returns the pickle data and serial number for the object specified by the given object id and version, if they exist; otherwise a KeyError is raised. iN(RtloadExRR(RRR((RtloadscCs9|iiz(|ii||}|o|SnWd|iiX|i djo t n|i iz|iiz||_ d|_Wd|iiX|i i||\}}}|iiz7|io |ii|||d|nd|_ Wd|iiXWd|i iX|||fS(Ni(RR^tacquireRhRRRRtreleaseR:RRR[R\R]RtdataR tvertstore(RRRRRR R((RRs8           cCs|iiz.|ii||}|dj o|SnWd|ii X|i i||}|djodSn|\}}}|djo|||fSn|iiz |ii|d|||Wd|ii X|||fS(NR(RR^RRht loadBeforeRR RRRR:RRtendR(RRR RRRR((RRs&       cCsY|iiz+|ii|}|dj o|SnWd|iiX|i i|S(sStorage API: return the version, if any, that modfied an object. If no version modified the object, return an empty string. N( RR^RRhtmodifiedInVersionRtvRRR:(RRR((RR s   cCsz|iotin|iiz>|ip#|ii|_|ii n|ii SWd|ii XdS(s,Storage API: return a new object identifier.N( RR@RRRYRRZR:tnew_oidstreversetpopR(R((Rtnew_oid.s   cCs>|djoti}n||d}|ii||S(sStorage API: pack the storage. Deviations from the Storage API: the referencesf argument is ignored; two additional optional arguments wait and days are provided: wait -- a flag indicating whether to wait for the pack to complete; defaults to true. days -- a number of days to subtract from the pack time; defaults to zero. iQN(RRRtdaysRR:tpackR4(RRt referencesfR4R((RR<s  cCsz|iolt|i}|i| }|i|4x;|D]3\}}t|to |n||i |Server callback to pass a list of changed (oid, serial) pairs.N(RRStextendtargs(RR((Rt serialnosscCs|ii|dS(s.Server callback to update the info dictionary.N(RRNRtdict(RR((Rtinfo scCs,|idjodSn|ii|dS(ssServer callback to invalidate an (oid, version) pair. This is called as part of cache validation. N(RRMRtdumpR(RR((RtinvalidateVerifyscCs|iizh}xf|D]^\}}}||ijo d|_ n|i i |||||i ||fh|