FINTECH • DATA SCRAPING • REST API
Ethiopia Forex Currency Exchange Monitor & API
A unified data extraction pipeline and REST API aggregating daily foreign exchange rates across Ethiopian commercial banks into a single standardized feed.
1. The Problem & Context in Ethiopia
Following recent macroeconomic reforms in Ethiopia, commercial banks set individual buying and selling exchange rates for major currencies (USD, EUR, GBP, AED, SAR). Because each financial institution publishes rates on disparate web pages with different table schemas, HTML layouts, and publication schedules, comparing rates across the banking sector requires manually checking dozens of websites.
Furthermore, local developers, import/export businesses, and Ethiopian diaspora communities lacked a clean, programmatically accessible REST API to query current rates without maintaining custom web scrapers themselves.
2. Architecture & Data Pipeline
To provide a resilient, reliable feed, the system is designed with a multi-stage data pipeline:
High-Level System Flow:
Bank HTML Scraping → DOM Normalization Engine → Currency Rate Sanitizer → In-Memory Cache → JSON REST Endpoints
The core pipeline performs the following steps:
- Automated Extraction: Scrapers periodically query public rate sheets from major Ethiopian commercial banks (Commercial Bank of Ethiopia, Awash Bank, Dashen Bank, Bank of Abyssinia, etc.).
- Schema Normalization: Bank tables vary wildly—some list rates as Buying/Selling/Cash, others use Transaction/Buying/Selling. The normalization engine translates each bank's terminology into a uniform JSON schema.
- Float Parsing & Sanitization: Cleans comma-separated strings, currency symbols, and inconsistent decimal formats into structured numeric floats.
- Caching & Rate Limiting: Implements rate caching to avoid hammering bank servers and guarantee sub-50ms API response times.
3. Technical Challenges & Solutions
Challenge A: Inconsistent and Shifting Bank Table Markup
Commercial bank websites often update their frontend styling, CMS themes, or table column orders without warning. A rigid scraping selector would break on minor markup changes.
Solution:
Implemented header-heuristic parsing. Instead of relying on brittle index positions (e.g. cells[2]), the parser dynamically inspects table header labels (matching keywords like "Buy", "Buying", "Sell", "Cash", "Transactional") to map columns to the appropriate fields dynamically.
Challenge B: Network Latency & Server Reliability
Local web hosts can experience periodic downtimes or slow connection handshakes during peak morning hours when exchange rates are posted.
Solution:
Configured resilient HTTP session handling with custom retry backoffs and timeout policies. If a bank website fails to respond within the threshold, the API serves the last known verified snapshot with a clear last_updated timestamp rather than crashing the request.
4. API Sample & Normalized JSON
The API provides standard endpoints for querying rates by bank, currency code, or aggregated market comparisons:
// GET /api/v1/rates/latest?currency=USD
{
"status": "success",
"base_currency": "ETB",
"target_currency": "USD",
"timestamp": "2026-09-08T06:00:00Z",
"banks": [
{
"bank_name": "Commercial Bank of Ethiopia",
"bank_code": "CBE",
"buying_rate": 138.4520,
"selling_rate": 141.2210,
"updated_at": "2026-09-08T05:30:12Z"
},
{
"bank_name": "Awash Bank",
"bank_code": "AWASH",
"buying_rate": 139.1050,
"selling_rate": 141.8870,
"updated_at": "2026-09-08T05:45:00Z"
}
]
}
5. Key Takeaways & Next Steps
Building ethiofx_api demonstrated the importance of fault-tolerant web scraping and robust schema normalization in financial data applications. Future enhancements include:
- Historical exchange rate trend charts and daily volatility alerts.
- A lightweight Telegram notification bot for instant rate change alerts.
- GraphQL endpoint support for granular client queries.