STEGInternship project2025
Network forecasting
A dashboard for forecasting network traffic and spotting unusual activity.
The problem
Network peaks and drops are easier to act on when teams can see what normal demand should look like. The challenge was earning a useful signal from exports with short histories, gaps, and inconsistent formatting.
What I built
The result was a modular analysis application where ingestion quality, forecasting evidence, and anomaly context are visible in the same workflow.
The result
Tuning lowered the evaluated hybrid XGBoost RMSE by 6.2%.
In context
Most evaluated histories covered only roughly twelve days of hourly observations.
Explore the technical details
Question & ownership
Can short, noisy hourly network histories produce forecasts and anomaly signals that are useful enough for operational monitoring?
I owned ingestion, exploratory analysis, feature construction, model comparison, temporal evaluation, anomaly logic, and the decision-facing Streamlit experience.
Data & validation
The source material consisted of hourly network-monitoring exports. Histories were short and varied in quality, which made ingestion and honest validation as important as model selection.
- Parse inconsistent exports and normalize timestamps to Africa/Tunis.
- Merge repeated uploads while deduplicating timestamps deterministically.
- Surface gaps and insufficient histories instead of silently fabricating features.
- Keep every published series anonymous and expose only aggregate evaluation evidence.
Key decisions
Validate forward in time
Random splitting leaks future patterns into training and produces an unrealistic estimate for a forecasting system.
Random train/test splits were rejected for model evaluation.
Keep a baseline ladder
Seasonal-naive and Holt-Winters baselines make it clear whether extra model complexity is buying meaningful improvement.
Jumping directly to LSTM would obscure whether the available history justified it.
Tune the nonlinear model on lag structure
Lag and calendar features let gradient boosting model nonlinear residual behavior without the serving cost or data hunger of a deeper sequence model.
SARIMAX and LSTM remained comparison candidates rather than assumed winners.
Overlay anomalies instead of declaring incidents
The model can identify unusual peaks and drops, but operational context is still required before calling them incidents.
Fully automatic incident classification would overstate what the available labels support.
Evaluation
A tuned hybrid XGBoost run reduced RMSE from 4.65 to 4.36 Mbps on the evaluated series. Cross-series results varied substantially, so the product exposes per-series evidence instead of a misleading global success score.
- Tuning lowered the evaluated hybrid XGBoost RMSE by 6.2%.
- Ten series produced usable evaluations; one short history was rejected rather than forced through the pipeline.
- Error varied materially across series, demonstrating that one aggregate metric would hide deployment risk.
Limitations & next steps
- Most evaluated histories covered only roughly twelve days of hourly observations.
- High error on some series limits any claim of universal forecasting reliability.
- An anomaly score signals unusual behavior but does not identify root cause or business impact.
- Collect longer histories and evaluate drift across operational seasons.
- Add prediction intervals and calibration checks to communicate uncertainty explicitly.
- Create an operator feedback loop for distinguishing incidents from benign traffic changes.
Sanitized case study