Note — Sep 08, 2019

Faced with a Data Deluge, Astronomers Turn to Automation

Basically, searching for stuff in space generates massive and ever growing amounts of data, which need to be processed (at least in part) in as close to realtime as possible so collaborating observatories can hand off coordinates and things to look for. Various groups are trying to implement AI to treat the data much faster, while looking at more parameters. Seems promising and includes questions around the black box problem.

“For each black hole merger signal that LIGO has detected that has been reported in publications, we can reconstruct all these parameters in two milliseconds,” Huerta says. In contrast, the traditional algorithms can take days to accomplish the same task. […]

When complete, the 8.4-meter LSST will be able to observe 10 square degrees of the sky at once (equivalent in size to 40 full moons), producing 15 to 20 terabytes of raw data each night—the same amount of data generated by the Sloan Digital Sky Survey over the course of a decade. […]

“If we are developing or constructing these next-generation instruments to study the universe in high fidelity, we also better design better algorithms to process this data.”