=wg^rdZddlmZddlZddlZddlZddlmZmZddl m Z ddl m Z ddl mZmZddlmZdd lmZmZmZd Zd ZGd d eZGddeZGddeZGddeZGddeZd#dZd$dZd#dZ d#dZ!d#dZ"d#dZ#Gdde$Z%dZ&Gdd e%Z'Gd!d"e$Z(y)%zSContains the main functions/classes for creating, maintaining, and using an index. )divisionN)timesleep) __version__) toc_loaders)pickle string_type) ensure_schema) _INT_SIZE _FLOAT_SIZE _LONG_SIZEMAINic eZdZy) LockErrorN)__name__ __module__ __qualname__C/var/www/horilla/myenv/lib/python3.12/site-packages/whoosh/index.pyrr1srrceZdZdZy) IndexErrorzGeneric index error.Nrrr__doc__rrrrr5srrceZdZdZddZy)IndexVersionErrorzRaised when you try to open an index using a format that the current version of Whoosh cannot read. That is, when the index you're trying to open is either not backward or forward compatible with this version of Whoosh. NcLtj||||_||_yN) Exception__init__versionrelease)selfmsgr!r"s rr zIndexVersionError.__init__@s 4%  rr)rrrrr rrrrr9s  rrceZdZdZy)OutOfDateErrorz^Raised when you try to commit changes to an index which is not the latest generation. Nrrrrr&r&Fsrr&ceZdZdZy)EmptyIndexErrorzIRaised when you try to work with an index that has no indexed terms. Nrrrrr(r(Lsrr(c\ddlm}|st}||}tj |||S)aConvenience function to create an index in a directory. Takes care of creating a FileStorage object for you. :param dirname: the path string of the directory in which to create the index. :param schema: a :class:`whoosh.fields.Schema` object describing the index's fields. :param indexname: the name of the index to create; you only need to specify this if you are creating multiple indexes within the same storage object. :returns: :class:`Index` r FileStorage)whoosh.filedb.filestorer+_DEF_INDEX_NAME FileIndexcreate)dirnameschema indexnamer+storages r create_inr4Ss.4 # '"G   GVY 77rcNddlm}|t}|||}t|||S)aDConvenience function for opening an index in a directory. Takes care of creating a FileStorage object for you. dirname is the filename of the directory in containing the index. indexname is the name of the index to create; you only need to specify this if you have multiple indexes within the same storage object. :param dirname: the path string of the directory in which to create the index. :param indexname: the name of the index to create; you only need to specify this if you have multiple indexes within the same storage object. rr*)readonly)r1r2)r,r+r-r.)r0r2r6r1r+r3s ropen_dirr7is-4# 'H5G WVy AArctjj|r! t||}|j dkDSy#t $rYywxYw)zReturns True if dirname contains a Whoosh index. :param dirname: the file path of a directory. :param indexname: the name of the index. If None, the default index name is used. r2F)ospathexistsr7latest_generationr()r0r2ixs r exists_inr@~sU ww~~g 'Y7B'')B. .     sA A Ac$|j|S)zDeprecated; use ``storage.index_exists()``. :param storage: a store.Storage object. :param indexname: the name of the index. If None, the default index name is used. ) index_exists)r3r2s rr=r=s    **rc8ddlm}||}t||S)aReturns a tuple of (release_version, format_version), where release_version is the release version number of the Whoosh code that created the index -- e.g. (0, 1, 24) -- and format_version is the version number of the on-disk format used for the index -- e.g. -102. You should avoid attaching significance to the second number (the index version). This is simply a version number for the TOC file and probably should not have been exposed in a public interface. The best way to check if the current version of Whoosh can open an index is to actually try to open it and see if it raises a ``whoosh.index.IndexVersionError`` exception. Note that the release and format version are available as attributes on the Index object in Index.release and Index.version. :param dirname: the file path of a directory containing an index. :param indexname: the name of the index. If None, the default index name is used. :returns: ((major_ver, minor_ver, build_ver), format_ver) rr*r9)r,r+r!)r0r2r+r3s r version_inrDs*4'"G 7i 00rc |t}|j|}|j|jfS#t$r(t j d}d|jfcYSwxYw)aReturns a tuple of (release_version, format_version), where release_version is the release version number of the Whoosh code that created the index -- e.g. (0, 1, 24) -- and format_version is the version number of the on-disk format used for the index -- e.g. -102. You should avoid attaching significance to the second number (the index version). This is simply a version number for the TOC file and probably should not have been exposed in a public interface. The best way to check if the current version of Whoosh can open an index is to actually try to open it and see if it raises a ``whoosh.index.IndexVersionError`` exception. Note that the release and format version are available as attributes on the Index object in Index.release and Index.version. :param storage: a store.Storage object. :param indexname: the name of the index. If None, the default index name is used. :returns: ((major_ver, minor_ver, build_ver), format_ver) N)r- open_indexr"r!rsysexc_info)r3r2r?es rr!r!sc*!  'I    * BJJ'' ! LLN1 aii  !s03.A$#A$ceZdZdZdZdZdZdZdZdZ dZ d Z d Z d Z d Zd ZdZdZddZdZddZddZy)Indexz3Represents an indexed collection of documents. cy)zCloses any open resources held by the Index object itself. This may not close all resources being used everywhere, for example by a Searcher object. Nrr#s rclosez Index.closes rch|j}|j|||jy)zAdds a field to the index's schema. :param fieldname: the name of the field to add. :param fieldspec: an instantiated :class:`whoosh.fields.FieldType` object. N)writer add_fieldcommit)r# fieldname fieldspecws rrRzIndex.add_fields& KKM Iy)  rcf|j}|j||jy)aRemoves the named field from the index's schema. Depending on the backend implementation, this may or may not actually remove existing data for the field from the index. Optimizing the index should always clear out existing data for a removed field. N)rQ remove_fieldrS)r#rTrVs rrXzIndex.remove_fields$ KKM y!  rcy)zReturns the generation number of the latest generation of this index, or -1 if the backend doesn't support versioning. r:rrNs rr>zIndex.latest_generationrc|S)zReturns a new Index object representing the latest generation of this index (if this object is the latest generation, or the backend doesn't support versioning, returns self). :returns: :class:`Index` rrNs rrefreshz Index.refreshs  rcy)zReturns True if this object represents the latest generation of this index. Returns False if this object is not the latest generation (that is, someone else has updated the index since you opened this object). TrrNs r up_to_datezIndex.up_to_dates rcy)zwReturns the last modified time of the index, or -1 if the backend doesn't support last-modified times. r:rrNs r last_modifiedzIndex.last_modifiedrZrct)zyReturns True if this index is empty (that is, it has never had any documents successfully written to it. NotImplementedErrorrNs ris_emptyzIndex.is_emptys "!rcy)z,Optimizes this index, if necessary. NrrNs roptimizezIndex.optimizes rc|j} |j|jS#|jwxYw)z\Returns the total number of documents, DELETED OR UNDELETED, in this index. )reader doc_count_allrOr#rs rrizIndex.doc_count_all!s1 KKM ??$ GGIAGGI 2Ac|j} |j|jS#|jwxYw)zGReturns the total number of UNDELETED documents in this index. )rh doc_countrOrjs rrnzIndex.doc_count,s0 KKM ;;= GGIAGGIrlc @ddlm}||jfd|i|S)zReturns a Searcher object for this index. Keyword arguments are passed to the Searcher object's constructor. :rtype: :class:`whoosh.searching.Searcher` r)Searcher fromindex)whoosh.searchingrprh)r#kwargsrps rsearcherzIndex.searcher6s" . @@@@rc|j} |j||jS#|jwxYw)zDReturns the total length of the field across all documents. )rh field_lengthrOr#rTrks rrvzIndex.field_length@s3 KKM >>), GGIAGGI 3Ac|j} |j||jS#|jwxYw)zFReturns the maximum length of the field across all documents. )rhmax_field_lengthrOrws rrzzIndex.max_field_lengthJs5 KKM %%i0 GGIAGGIrxNct)aReturns an IndexReader object for this index. :param reuse: an existing reader. Some implementations may recycle resources from this existing reader to create the new reader. Note that any resources in the "recycled" reader that are not used by the new reader will be CLOSED, so you CANNOT use it afterward. :rtype: :class:`whoosh.reading.IndexReader` rb)r#reuses rrhz Index.readerTs "!rc t)zkReturns an IndexWriter object for this index. :rtype: :class:`whoosh.writing.IndexWriter` rb)r#rss rrQz Index.writer`s "!rcl|j}|j||||jyN)rt)rQdelete_by_termrS)r#rTtextrtrVs rrzIndex.delete_by_termgs+ KKM D8<  rcj|j}|j|||jyr)rQdelete_by_queryrS)r#qrtrVs rrzIndex.delete_by_queryls) KKM !h/  rr)rrrrrOrRrXr>r\r^r`rdrfrirnrtrvrzrhrQrrrrrrLrLsb     "  A "" rrLctd|D}tj|}tj|}t}|D]}|j dr|j |} |j |} | r0t | jd|k7sW|j|i| sl| jd} | |vs|j||D]} |j|y#t$rY#wxYw)Nc3<K|]}|jywr segment_id).0ss r zclean_files..zsA1 As.rF) setTOC_pattern_segment_pattern startswithmatchintgroupadd delete_fileOSError) r3r2gensegmentscurrent_segment_names tocpattern segpatterntodeletefilenametocmsegmnames r clean_filesrts  AAAi(J%%i0JuH '   s # )) 4::a=!S( X& ::a=D00 X& '     )   s$C88 DDceZdZdefdZeefdZdZdZdZ dZ dZ d Z dd Z d Zd Zd ZdZedZedZedZeddZddZy)r.Nc0ddlm}t||std|zt|tstd|z|r t |}||_||_||_tj|j |j|jy)Nr)Storagez%r is not a Storage objectzindexname %r is not a stringr1) r,r isinstance ValueErrorr r r3_schemar2rread)r#r3r1r2rs rr zFileIndex.__init__s}3'7+9GCD D)[1;iGH H "6*F  " t~~dllCrcDtj|||||||Sr)rr/)clsr3r1r2s rr/zFileIndex.creates! 7FI.7FI..rch|jjd|jd|jdS)N(z, )) __class__rr3r2rNs r__repr__zFileIndex.__repr__s&#~~66#||T^^= =rcyrrrNs rrOzFileIndex.closes rcVtj|j|jSr)r_latest_generationr3r2rNs rr>zFileIndex.latest_generations%%dllDNNCCrc|j}tj|j|}|jj |Sr)r>r _filenamer2r3 file_modified)r#rrs rr`zFileIndex.last_modifieds9$$&==5||))(33rcNt|jjdk(S)Nr)len _read_tocrrNs rrdzFileIndex.is_emptys4>>#,,-22rc L|jdi|}|jdy)NT)rfr)rQrS)r#rsrVs rrfzFileIndex.optimizes# DKK !& ! $rc L|dkDrddlm}||fd|i|Sddlm}||fi|S)NrFr)MpWriterprocs) SegmentWriter)whoosh.multiprocrwhoosh.writingr)r#rrsrrs rrQzFileIndex.writers4 19 1D888 8 4 00 0rcX|jj|jdz|zS)z_Returns a lock object that you can try to call acquire() on to lock the index. _)r3lockr2)r#rs rrzFileIndex.locks' ||  #!5!<==rcntj|j|j|jS)Nr)rrr3r2rrNs rrzFileIndex._read_tocs#xx dnnT\\xJJrc6|jjSr)rrrNs r _segmentszFileIndex._segmentss~~(((rc6|jjSr)rr1rNs r_current_schemazFileIndex._current_schemas~~&&&rc"|jSr)rrNs rr1zFileIndex.schemas##%%rc6|jjSr)rr"rNs rr"zFileIndex.release~~'''rc6|jjSr)rr!rNs rr!zFileIndex.versionrrc ddlm m}m}i t |dk(r-|j D]}|j S|r4|jD cgc]\}} | } }} td| D fd} t |dk(r0| |dj D]}|j S|D cgc] } | |  } } || j D]}|j Scc} }wcc} w#j D]}|j wxYw)Nr) SegmentReader MultiReader EmptyReaderc3@K|]}|j|fywr generation)rrks rrz$FileIndex._reader..sE 3EscX|j}|vr |}|=|S|S)Nrr)segmentsegidrkrrreusabler1r3s r segreaderz$FileIndex._reader..segreadersF**,H$ A H(&'4>@@rrFr) whoosh.readingrrrrvaluesrO leaf_readersdict)rr3r1rrr|rrrkrreadersrrrrs `` ` @@r_readerzFileIndex._readers@ KJ& 8}!#6*B__&   ?*/););)=>A1>>EWEE  @ @8}!!!-__&   >FF'9W-FF"7zB__&   9?2G__&   s5DD# D/6D DD D D'Ec:d}|dkDrP |j}|j|j|j|j|j |Syy#t $r1tjd}|dz}|dkr|tdYnwxYw|dkDrF)N r)r|rFg?) rrr3r1rrIOErrorrHrIr)r#r|retriesinforJs rrhzFileIndex.readersk ~~'||DLL$++t}}$(OO5$BB    LLN1%1 a<Gd   ksA A7BB)rFr)rrrr-r classmethodr/rrOr>r`rdrfrQrrrrpropertyr1r"r!rrhrrrr.r.s'+D$/>//= D 4 3 1>K)'&&((((,,\rr.ceZdZdZeefdZedZedZ edZ edZ ee fdZ ed d Zd Zy) rzObject representing the state of the index after a commit. Essentially a container for the index's schema and the list of segment objects. cJ||_||_||_||_||_yr)r1rrr!r")r#r1rrr!r"s rr z TOC.__init__8s%   $  rcd|d|dS)Nrz.tocr)rr2rs rrz TOC._filename@s (#..rc2tjd|zS)Nz^_%s_([0-9]+).toc$recompilerr2s rrz TOC._patternDszz.:;;rc2tjd|zS)Nz(%s_[0-9a-z]+)[.][A-Za-z0-9_.]+rrs rrzTOC._segment_patternHszz;iGHHrc|j|}d}|D]:}|j|}|stt|j d|}<|S)Nr:rF)rrmaxrr)rr3r2patternmxrms rrzTOC._latest_generationLsT,,y)  .H h'AQWWQZ"- . rct|}d|z}|D]%}|j|s|j|'||gd}|j||y)Nz_%s_r)r rrwrite)rr3r1r2prefixrtocs rr/z TOC.createWsav&)# .H""6*##H- . &"a  '9%rNcJ |(|j||}|dkrtd|d||j||}|j| fd}|dt|dt |dt  jdk(s td  j} j j jf}|tk7r/|tvrt|} | |||\}} ntd |z||r jn#tj j!}t#|} j} || k(sJ j} j%} j'||| ||| S) NrzIndex z does not exist in cXj}||k7rtd|d|d|y)Nz3Index was created on different architecture: saved z = z, this computer = ) read_varintr)rtargetszstreams r check_sizezTOC.read..check_sizeqs6##%BV| $("f"677rrlongfloatz"Number misread: byte order problemzCan't read format %s)r!r")rr(r open_filer r r read_intrr_CURRENT_TOC_VERSIONrr skip_stringrloads read_stringr read_picklerO)rr3r2rr1 tocfilenamerr!r"loaderr index_genrrs @rrzTOC.reades ;(()>mmIs3 "";/ 7 5)$6:&7K( F*AB B//#%%'););)=%%') * *+%$W-#)&#vw#G '(>(H(/11 ""$f&8&8&:;"6*F)I)# ##!A))+H 68S'7KKrc vt|j}|j|j||j}|dt }|j |}|jt|jt|jt|jd|jttddD]}|j| |jtj |d|j|j|jd|j-|j.|j1|j3||d y#tj"$r|j%D]\}} tj | #tj"$r5t'j(d} tj"| d|d| t*$r+t'j(d} t+| d|d| wxYwwxYw) NrrrF =rT)safe)r r1cleanrrr create_file write_varintr r r write_intrr write_stringrdumps PicklingErroritemsrHrI TypeError write_picklerrO rename_file) r#r3r2r1r  tempfilenamernumrTfieldrJs rrz TOC.writest{{+ nnY@ "-tv6 $$\2I&J'K( -.r? %C    $ %     VQ 7 8 )DMM*  L+DA)## $*LLN H 5HLL'++S q)A ..Q 5/QRR H q)A#!Y$FGGH  H  s%'%E88)H8"F87H88A:H22H8)NN)rrrrrrr rrrrrr-r/rrrrrrr3s .{//<<II/> & &5L5Ln(Brrr)NFN))r __future__ros.pathr;rrHrrwhooshr whoosh.legacyr whoosh.compatrr whoosh.fieldsr whoosh.systemr r r r-rrrrrr&r(r4r7r@r=rDr!objectrLrr.rrrrr)s8 %-'<<     Z j8,B*$+14!DXFXz@ZZ~RB&RBr