Fool’s Gold: Data Mining Digs Up Explosive Errors
Recent studies reveal the prevalence of poor-quality data, exacerbated by increased use of machine learning that allows users to dredge far bigger datasets and identify spurious correlations.
A review of 100 major psychology studies, for instance, found that only 36 percent had statistical significance. Over half the alien planets identified by Nasa’s Kepler telescope turned out to be stars. And in preclinical cancer research, a mere six out of 53 breakthrough studies were found to be reproducible. Quantitative finance does not fare much better.
“It’s a gigantic problem—spurious results are the norm,” says Zak David, co-founder of analytics firm Mile 59, and former engineer of high
Only users who have a paid subscription or are part of a corporate subscription are able to print or copy content.
To access these options, along with all other subscription benefits, please contact info@waterstechnology.com or view our subscription options here: http://subscriptions.waterstechnology.com/subscribe
You are currently unable to print this content. Please contact info@waterstechnology.com to find out more.
You are currently unable to copy this content. Please contact info@waterstechnology.com to find out more.
Copyright Infopro Digital Limited. All rights reserved.
You may share this content using our article tools. Printing this content is for the sole use of the Authorised User (named subscriber), as outlined in our terms and conditions - https://www.infopro-insight.com/terms-conditions/insight-subscriptions/
If you would like to purchase additional rights please email info@waterstechnology.com
Copyright Infopro Digital Limited. All rights reserved.
You may share this content using our article tools. Copying this content is for the sole use of the Authorised User (named subscriber), as outlined in our terms and conditions - https://www.infopro-insight.com/terms-conditions/insight-subscriptions/
If you would like to purchase additional rights please email info@waterstechnology.com
More on Data Management
Northern Trust building internal cloud data ‘marketplace’
Using a mix of in-house expertise and third-party technologies, the firm has constructed a cloud-based data mesh that gives internal staff access to proprietary datasets and analytical tools to deliver greater insights into client activity.
FactSet looks to build on portfolio commentary with AI
Its new solution will allow users to write attribution summaries more quickly and adds to its goal of further accelerating discoverability, automation, and innovation.
How Ally found the key to GenAI at the bottom of a teacup
Risk-and-tech chemistry—plus Microsoft’s flexibility—has seen the US lender leap from experiments to execution.
The IMD Wrap: Beginning of the end for data audits?
This week, there’s exciting news for data bean-counters in the form of a partnership between two vendors that could change the way we view and track data usage and audits.
Symphony boosts Cloud9 voice offerings with AI
The messaging and collaboration platform builds on Cloud9’s capabilities as it embraces the AI wave in what CEO Brad Levy calls “incremental” steps.
MSCI counts the cost of bank M&A, looks ahead to custom indexes, AI
Cancellations of overlapping contracts following a bank merger put a dent in MSCI’s earnings, but management remains upbeat about the potential of recent acquisitions and new technology developments.
Nasdaq reshuffles tech divisions post-Adenza
Adenza is now fully integrated into the exchange operator’s ecosystem, bringing opportunities for new business and a fresh perspective on how fintech fits into its strategy.
This Week: HKEx's new derivatives platform; GoldenSource; Quonian-SimCorp, and more
A summary of some of the latest financial technology news.
Most read
- Chris Edmonds takes the reins at ICE Fixed Income and Data Services
- Women in Technology & Data Awards 2024: All the winners and why they won
- DTCC urges affirmation focus ahead of T+1 move