A Transformer-Based Framework for Context-Aware ESG Corpus Construction

Abstract

This paper develops a transformer-based method for constructing a context-aware Environmental, Social, and Governance (ESG) finance corpus. The proposed framework uses attention-based relevance scores to identify ESG terms and sentences according to their surrounding textual context. The method is first illustrated through a probabilistic ESG term matrix and then evaluated using a validation sample of SEC EDGAR 10-K filings from 73 S&P 500 firms across 11 GICS sectors. The resulting balanced corpus contains 1,456 ESG sentences, with environmental, social, and governance content representing 34.8%, 35.1%, and 30.1% of the sample, respectively. Validation evidence indicates that the attention-based procedure performs well relative to alternative corpus construction methods. Ten-fold cross-validation produces an overall F1-score of 87.7%, while corpus-derived ESG intensity scores correlate positively with alternative ESG ratings. The attention-based method also outperforms keyword matching, TF-IDF filtering, and a supervised BERT benchmark on both classification performance and external rating alignment.