=wg!`dZddlmZddlmZmZddlmZmZm Z dZ dZ dZ Gdd e Zy ) aThis module contains low-level functions and a high-level class for parsing the prolog file "wn_s.pl" from the WordNet prolog download into an object suitable for looking up synonyms and performing query expansion. http://wordnetcode.princeton.edu/3.0/WNprolog-3.0.tar.gz ) defaultdict)iterkeys text_type)SchemaIDSTOREDctt}tt}|D]}|jds|dd}t|d|j d}|j d}||dzd}|j d}|d|j }|j s||j|||j|||fS)zfParses the WordNet wn_s.pl prolog file and returns two dictionaries: word2nums and num2words. zs(N,')rlist startswithintfindlowerisalphaappend)f word2nums num2wordslinenumqtwords J/var/www/horilla/myenv/lib/python3.12/site-packages/whoosh/lang/wordnet.py parse_filer)s D!ID!I$t$ ABx$ #'( YYs^BFG} YYs^CRy ||~ $s##d#$" i cttt}|j||}|j }t |D]+}t |||}|jt||-|j|S)zCreates a Whoosh index in the given storage object containing synonyms taken from word2nums and num2words. Returns the Index object. )rsyns indexname) rrr create_indexwriterrsynonyms add_documentrcommit) storager"rrschemaixwrr s r make_indexr,Esv & )F   f  :B A#8 9d3 IdO$78HHJ Irc||}t}|D]}|j||}||vr|j|t|S)zzUses the word2nums and num2words dicts to look up synonyms for the given word. Returns a list of synonym strings. )setunionremovesorted)rrrkeysr keys rr%r%UsT T?D 5D*zz)C.)* t| D $<rcVeZdZdZdZedZedZed dZd dZ dZ y) Thesaurusa Represents the WordNet synonym database, either loaded into memory from the wn_s.pl Prolog file, or stored on disk in a Whoosh index. This class allows you to parse the prolog file "wn_s.pl" from the WordNet prolog download into an object suitable for looking up synonyms and performing query expansion. http://wordnetcode.princeton.edu/3.0/WNprolog-3.0.tar.gz To load a Thesaurus object from the wn_s.pl file... >>> t = Thesaurus.from_filename("wn_s.pl") To save the in-memory Thesaurus to a Whoosh index... >>> from whoosh.filedb.filestore import FileStorage >>> fs = FileStorage("index") >>> t.to_storage(fs) To load a Thesaurus object from a Whoosh index... >>> t = Thesaurus.from_storage(fs) The Thesaurus object is thus usable in two ways: * Parse the wn_s.pl file into memory (Thesaurus.from_*) and then look up synonyms in memory. This has a startup cost for parsing the file, and uses quite a bit of memory to store two large dictionaries, however synonym look-ups are very fast. * Parse the wn_s.pl file into memory (Thesaurus.from_filename) then save it to an index (to_storage). From then on, open the thesaurus from the saved index (Thesaurus.from_storage). This has a large cost for storing the index, but after that it is faster to open the Thesaurus (than re-parsing the file) but slightly slower to look up synonyms. Here are timings for various tasks on my (fast) Windows machine, which might give an idea of relative costs for in-memory vs. on-disk. ================================================ ================ Task Approx. time (s) ================================================ ================ Parsing the wn_s.pl file 1.045 Saving to an on-disk index 13.084 Loading from an on-disk index 0.082 Look up synonyms for "light" (in memory) 0.0011 Look up synonyms for "light" (loaded from disk) 0.0028 ================================================ ================ Basically, if you can afford spending the memory necessary to parse the Thesaurus and then cache it, it's faster. Otherwise, use an on-disk index. c.d|_d|_d|_y)N)w2nn2wsearcher)selfs r__init__zThesaurus.__init__s rcD|}t|\|_|_|S)aCreates a Thesaurus object from the given file-like object, which should contain the WordNet wn_s.pl file. >>> f = open("wn_s.pl") >>> t = Thesaurus.from_file(f) >>> t.synonyms("hail") ['acclaim', 'come', 'herald'] )rr7r8)clsfileobjthess r from_filezThesaurus.from_files#u'0$( rct|d} |j||jS#|jwxYw)zCreates a Thesaurus object from the given filename, which should contain the WordNet wn_s.pl file. >>> t = Thesaurus.from_filename("wn_s.pl") >>> t.synonyms("hail") ['acclaim', 'come', 'herald'] rb)openr@close)r=filenamers r from_filenamezThesaurus.from_filenames4 4  ==# GGIAGGIs /Acb|}|j|}|j|_|S)aJCreates a Thesaurus object from the given storage object, which should contain an index created by Thesaurus.to_storage(). >>> from whoosh.filedb.filestore import FileStorage >>> fs = FileStorage("index") >>> t = Thesaurus.from_storage(fs) >>> t.synonyms("hail") ['acclaim', 'come', 'herald'] :param storage: A :class:`whoosh.store.Storage` object from which to load the index. :param indexname: A name for the index. This allows you to store multiple indexes in the same storage object. r!) open_indexr9)r=r(r"r?indexs r from_storagezThesaurus.from_storages1"u""Y"7(  rc|jr |js tdt|||j|jy)aCreates am index in the given storage object from the synonyms loaded from a WordNet file. >>> from whoosh.filedb.filestore import FileStorage >>> fs = FileStorage("index") >>> t = Thesaurus.from_filename("wn_s.pl") >>> t.to_storage(fs) :param storage: A :class:`whoosh.store.Storage` object in which to save the index. :param indexname: A name for the index. This allows you to store multiple indexes in the same storage object. zNo synonyms loadedN)r7r8 Exceptionr,)r:r(r"s r to_storagezThesaurus.to_storages3xxtxx01 17Itxx:rc|j}|jr|jj|dSt|j|j |S)zReturns a list of synonyms for the given word. >>> thesaurus.synonyms("hail") ['acclaim', 'come', 'herald'] )rr )rr9documentr%r7r8)r:rs rr%zThesaurus.synonymssJzz| ====))t)4V< <DHHdhh5 5rN)THES) __name__ __module__ __qualname____doc__r; classmethodr@rFrJrMr%rrr5r5dsS3j     *;& 6rr5N)rT collectionsr whoosh.compatrr whoosh.fieldsrrrrr,r%objectr5rVrrr[s58$-,, 8  N6N6r