The ARTFL data compression project
Pages 967 - 985
Abstract
This paper reports on a variety of compression algorithms developed in the context of a project to put all the data files for a full-text retrieval system on a CD-Rom. In the context of inexpensive pre-processing, a text compression algorithm is presented that is based on Markov-modeled Huffman coding on an extended alphabet. Data structures are examined for facilitating random access into the compressed text. In addition, new algorithms are presented for compression of word indices, both the dictionaries (word lists), and the text pointers (concordances).
References
[1]
T. Bell, et. al., "Modeling for Text Compression", ACM Computing Surveys, 21 (1989) 557--592.
[2]
A. Bookstein, S.T. Klein, "Compression, Information Theory and Grammars: A Unified Approach", ACM Transactions on Information Systems, 8 (1990) 27--49.
[3]
A. Bookstein, S.T. Klein, "Using Bitmaps for Medium Sized Information Retrieval Systems", to appear in Information Processing & Management, 26 (1990).
[4]
A. Bookstein, J. Handley, "Comparison of Analytic Models and CHRT for Database Subsettings", OCLC Working Paper, Office of Research, OCLC, 1990.
[5]
P. Bratley, Y. Choueka, "Processing truncated terms in document retrieval systems", Inf. Processing & Management, 18 (1982) 257--266.
[6]
Y. Choueka, A.S. Fraenkel, S.T. Klein, E. Segal, "Improved techniques for processing queries in full-text systems", Proc. 10th ACM-SIGIR Conf., New Orleans (1987) 306--315.
[7]
Y. Choueka, A.S. Fraenkel, S.T. Klein, "Compression of concordances in full-text retrieval systems", Proc. 11-th ACM-SIGIR Conf., Grenoble (1988) 597--612.
[8]
E.M. Cichocki, S.M. Ziemer, "Design considerations for CD-ROM retrieval software", J. Amer. Soc. Inf. Sc., 39 (1988) 43--46.
[9]
D.H. Davies, "The CD-ROM medium", J. Amer. Soc. Inf. Sc., 39 (1988) 34--42.
[10]
C. Faloutsos, S. Christodoulakis, "Signature files: An access method for documents and its analytical performance evaluation", ACM Transactions on Information Systems, 2 (1984) 267--288.
[11]
D.A. Lelewer, D.S. Hirschberg, "Data Compression", ACM Computing Surveys, 19 (1987) 261--296.
[12]
S.T. Klein, et. al., "Storing Text Retrieval Systems on CD-Rom: Compression and Encryption Consideration", ACM Transactions on Information Systems, 7(3) (1989) 230--245.
Information & Contributors
Information
Published In
Publisher
LE CENTRE DE HAUTES ETUDES INTERNATIONALES D'INFORMATIQUE DOCUMENTAIRE
Paris, France
Publication History
Published: 02 April 1991
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 47Total Downloads
- Downloads (Last 12 months)37
- Downloads (Last 6 weeks)5
Reflects downloads up to 25 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in