X XFediYour place in the Fediverse
XFEDI

#DataEngineering

ActivityPub

Public posts with this hashtag.

↳ In reply to a post

New study extracts dominant influence routes from massive climate datasets. It shows causal pathways between teleconnections.
The computing framework avoids ML. Phases include:
• Gridded data is transformed and stored in relational and graph databases for fast geospatial retrieval.
• The pipeline attacks dimensionality through spatial clustering.
• Causal processing: one representative time series per cluster is computed by the SLARAC method.
• A Yen’s k-shortest path algorithm extracts the most relevant causal chains.

Cruz, J. E. C., da Rocha, V. R., de Andrade, F. M., & Quiles, M. G. (2026). "A multivariate causal chain detection methodology assessed on the ENSO modulation of the Northeastern Brazil precipitation" https://doi.org/10.1007/s41060-026-01322-3

#ENSO #ElNiño #causality #climate #weather #SLARAC #networks #dataEngineering #Brazil

Three phase approach - data transformation and ingestion (blue), clustering (green), and causal processing (orange)

from Cruz, J. E. C., da Rocha, V. R., de Andrade, F. M., & Quiles, M. G. (2026). "A multivariate causal chain detection methodology assessed on the ENSO modulation of the Northeastern Brazil precipitation" https://doi.org/10.1007/s41060-026-01322-3
↩ Reply ♡ 0 · ↻ 0 ↗ Original

Explore the hidden costs of real-time data pipelines and learn why latency, data freshness, correctness, and recovery matter in Kafka and streaming architecture https://hackernoon.com/the-hidden-cost-of-real-time-data-why-faster-pipelines-dont-always-deliver-better-decisions #dataengineering

hackernoon.comThe Hidden Cost of Real-Time Data: Why Faster Pipelines Don't Always Deliver Better Decisions | HackerNoonExplore the hidden costs of real-time data pipelines and learn why latency, data freshness, correctness, and recovery matter in Kafka and streaming architecture
↩ Reply ♡ 0 · ↻ 0 ↗ Original

And we're live for Oct!

Title: Linting and data viz silos
Speakers: Etienne Bacher and Neil Pettinger
Date: Wednesday 21st October 2026, 6.00PM - 7.00PM
Location: 2.11 Appleton Tower, The University of Edinburgh
Register: https://luma.com/nulvgls1

Looking forward to seeing you there.

#RStats #DataScience #DataEngineering #Tech #meetup #Edinburgh #Scotland

@mikerspencer @etiennebacher

luma.comOctober 2026: Linting and data viz silos · LumaTitle: Linting and data viz silos Speakers: Etienne Bacher and Neil Pettinger Date: Wednesday 21st October 2026, 6.00PM - 7.00PM Location: 2.11 Appleton Tower,…
↩ Reply ♡ 0 · ↻ 0 ↗ Original