Theoretical Foundations of Hybrid Information Retrieval Based on a Context-Aware Dynamic Alpha Model
Article Sidebar
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Main Article Content
Abstract
This paper presents the theoretical foundations of the Context-Aware Dynamic Weighting Model (CADWM) - a hybrid information retrieval framework in which the interpolation coefficient α between a lexical scorer (BM25) and a semantic model (BERT-based similarity) is predicted per-query by a lightweight neural network. Rather than treating α as a global constant, CADWM formalises it as a function α(q, u, c) of the query q, the user profile u, and the session context c. The paper covers the formal model definition, the 26-dimensional feature space decomposition, the neural architecture (AlphaNet), the training objective with MSE loss and L2 regularisation, and a full complexity and latency analysis. Theoretical arguments are given for why a compact two-hidden-layer network is sufficient for this regression task and why MSE regression on grid-searched targets is preferable to a direct ranking loss under data-scarce conditions. The model is illustrated on the task of Uzbek legal document retrieval, where static α selection leads to systematic sub-optimality across heterogeneous query intents.
Article Details
References
Robertson, S., & Zaragoza, H. (2009). The Probabilistic Relevance Framework: BM25 and Beyond. Foundations and Trends in Information Retrieval, 3(4), 333–389.
Mitra, B., & Craswell, N. (2018). An Introduction to Neural Information Retrieval. Foundations and Trends in Information Retrieval, 13(1), 1–126.
Gao, L., Ma, X., Lin, J., & Callan, J. (2021). Complementing Lexical Retrieval with Semantic Residual Embedding. In Proceedings of ECIR 2021, pp. 234–250.
Lin, J., & Ma, X. (2021). A Few Brief Notes on DeepImpact, COIL, and a Conceptual Framework for Information Retrieval Techniques. arXiv:2106.14807.
Mallia, A., Khattab, O., Suel, T., & Tonellotto, N. (2021). Learning Passage Impacts for Inverted Indexes. In Proceedings of SIGIR 2021, pp. 1723–1727.
Luan, Y., Eisenstein, J., Toutanova, K., & Collins, M. (2021). Sparse, Dense, and Attentional Representations for Text Retrieval. Transactions of the ACL, 9, 329–345.
Ma, X., Zhang, P., & Lin, J. (2022). Query-Level Adaptive Weighting for Hybrid Retrieval. In Proceedings of SIGIR 2022, pp. 2100–2110.
Nogueira, R., & Cho, K. (2020). Passage Re-ranking with BERT. arXiv:1901.04085.
Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., & Yih, W. (2020). Dense Passage Retrieval for Open-Domain Question Answering. In Proceedings of EMNLP 2020, pp. 6769–6781.
Reimers, N., & Gurevych, I. (2019). Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In Proceedings of EMNLP-IJCNLP 2019, pp. 3982–3992.
Zhao, T., Zhang, X., & Lu, W. (2021). SPARTA: Efficient Open-Domain Question Answering via Sparse Transformer Matching Retrieval. In Proceedings of NAACL 2021, pp. 565–575.
Cormack, G. V., Clarke, C. L. A., & Buettcher, S. (2009). Reciprocal Rank Fusion Outperforms Condorcet and Individual Rank Learning Methods. In Proceedings of SIGIR 2009, pp. 758–759.
Formal, T., Piwowarski, B., & Clinchant, S. (2021). SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking. In Proceedings of SIGIR 2021, pp. 2288–2292.
Agichtein, E., Brill, E., & Dumais, S. (2006). Improving Web Search Ranking by Incorporating User Behavior Information. In Proceedings of SIGIR 2006, pp. 19–26.
Vulić, I., & Moens, M.-F. (2015). Monolingual and Cross-Lingual Information Retrieval Models Based on (Bilingual) Word Embeddings. In Proceedings of SIGIR 2015, pp. 363–372.
Mansurov, J., & Mansurov, A. (2021). UzBERT: A BERT-based Language Model for Uzbek. arXiv:2109.12202.
Hornik, K., Stinchcombe, M., & White, H. (1989). Multilayer Feedforward Networks are Universal Approximators. Neural Networks, 2(5), 359–366.
Kingma, D. P., & Ba, J. (2015). Adam: A Method for Stochastic Optimization. In Proceedings of ICLR 2015.
Bartlett, P. L., Harvey, N., Liaw, C., & Mehrabian, A. (2019). Nearly-tight VC-Dimension and Pseudodimension Bounds for Piecewise Linear Neural Networks. Journal of Machine Learning Research, 20(63), 1–17.
Jacobs, R. A., Jordan, M. I., Nowlan, S. J., & Hinton, G. E. (1991). Adaptive Mixtures of Local Experts. Neural Computation, 3(1), 79–87.