Sentiment Analysis on Financial News

Overview

An end-to-end sentiment analysis service that classifies financial news headlines as positive, negative or neutral in both Turkish and English. The system automatically collects headlines from live news sources, filters out non-financial content and summarises market sentiment in a single interface.

The Need

Financial news flows fast and in high volume; tracking market sentiment by hand does not scale. Existing sentiment models are largely English-centric and rarely sustain their benchmark performance on real headlines. The goal was a system that works reliably in two languages and is validated on real-world data.

Solution Architecture

1. Data collection — Turkish RSS sources such as Bloomberg HT, Dünya, Ekonomim and AA Economy; English sources such as Yahoo Finance, CNBC, MarketWatch and the Financial Times; plus the Finnhub and Newsdata.io APIs.
2. Off-topic filter — A binary classifier that removes celebrity, sports and health headlines before sentiment analysis.
3. Sentiment model — xlm-roberta-base, fine-tuned on Turkish and English financial text.
4. Service layer — /predict and /news endpoints on FastAPI, with a web interface that visualises sentiment distribution and scores user input in real time.

Modelling

Work began with a baseline to establish a reference point: TF-IDF and logistic regression, 0.69 macro-F1 on Financial PhraseBank. XLM-RoBERTa was then fine-tuned on Apple Silicon (MPS) using a dataset extended with ITU KAP headlines and a Turkish translation of PhraseBank, reaching 0.79 macro-F1 on the mixed test set (English 0.85, Turkish 0.78).

Two safeguards protect the integrity of the evaluation:

• Leakage control — An English sentence and its Turkish translation are assigned to the same group, preventing information from leaking between training and test sets.
• Balanced class weights — The square root of the balancing weights is used instead of the standard scheme, preventing the neutral class from being suppressed.

Validation on Real Data

When tested on 129 Turkish headlines collected from live sources and labelled by hand, performance fell to 0.53 macro-F1. Analysis identified the cause as domain shift: the training data consisted of KAP disclosures describing concrete financial events, while a large share of real headlines signal no direction at all.

To close this gap, a neutral decision threshold calibrated through cross-validation was introduced:

The off-topic filter had to be trained on only 46 examples. Where TF-IDF fell short at this scale (0.69), a classifier built on XLM-R embeddings reached 0.82; 294 additional headlines labelled by a language model raised it to 0.84.

Engineering Standards

• Data sources, preparation, modelling and the service are designed as independent modules.
• A dedicated labelling interface was built to accelerate validation on real data.
• The API and end-to-end flow are covered by 10 automated tests.

Roadmap

Sector and topic classification (monetary policy, energy, banking, crypto), and a model retrained on real headlines to replace the neutral threshold.