Conflicting numbers, information gaps: Following NC’s COVID-19 data can be frustrating

COVID-19 information provided on North Carolina’s website might seem easy to understand at first, but a deeper look reveals numbers that don’t always add up.

Lagging test results means the agency backfills test information as it comes in. In a written explanation on its website, the state Department of Health and Human Services calls the daily total test figure “a dynamic number.”

“Those numbers change because we are getting updated information and sometimes it can be backdated,” said Jessie Tenenbaum, chief data officer at DHHS.

Problems with test reports from LabCorp, a national diagnostic company based in Burlington, have produced a distorted view of how many tests were performed in North Carolina this year and how many people tested positive this month. LabCorp handles about half the state’s COVID-19 tests, DHHS says.

DHHS said a report of 2,500 new cases on Saturday was inflated by about 1,000 cases that should have been reported earlier in August.

In an email Monday, Pattie Kushner, LabCorp’s chief communications officer, told The News & Observer the state received only partial files for three dates earlier in August. She said no one from the company was available for an interview. No other state had received partial files, she wrote.

Earlier in August, LabCorp and the state reported that the lab had mixed in some out-of-state results with North Carolina results, which inflated the total tests performed.

Daily percent positive rates sometimes change after they’re reported, too. And that percentage cannot be calculated independently using the information available on the dashboard.

The data collection is important: State officials use information on total tests performed, new cases, percent positive tests, and hospitalizations to decide on actions to curb viral spread.

When DHHS announced that it had discovered the first LabCorp error that resulted in an overcount of total tests by more than 221,000, it magnified the fine print on how the agency collects and reports information.

The DHHS public-facing data dashboard is built from many streams of information, some of it delivered electronically and automatically fed into reporting system, and some sent by fax and typed into the system.

The race or ethnicity for people diagnosed with COVID-19 is missing in nearly 93,000 cases.

DHHS said that information is not usually supplied with lab results. Local health departments try to find that information when they contact people who test positive, the agency said in an email.

The new test totals DHHS reports daily come from numbers that labs submit by fax. Most of the other numbers, including most of the patient-specific test results, come from automatic electronic feeds.

The “percent positive” data DHHS reports each day uses only electronic reports, which account for about 80% of the tests. That’s why using the daily total tests and the day’s new cases to calculate the percent positive won’t match the number the state report. DHHS says it is working to cut back on reports labs file manually.

Tenenbaum said in an interview with The News & Observer that the agency’s readiness to announce mistakes and to correct the information should give the public confidence that it is working to get the numbers right.

“All the data is highly complex,” she said. “We’re augmenting and improving the systems. If we weren’t finding the problems, I would be more suspicious. DHHS is doing their very best to get the most accurate data. Where we find that’s not the case, we’re being completely transparent with the public.”

Frustrating fluctuations

The information DHHS shows is constantly updated and corrected, with most of those changes coming without any special announcements. Labs are asked to get their test information to DHHS by 7:30 a.m. each day. Results that miss the deadline are added the next day.

All those fluctuating figures frustrate Brandi Holland, a teacher in Belhaven, who regularly scrutinizes the data and tunes in to the news conferences where Gov. Roy Cooper and Dr. Mandy Cohen, DHHS secretary, provide updates on the state’s response to the pandemic. The 221,000 test error eroded her confidence in the state’s data, she said.

“I really wanted to become informed about the numbers,” Holland said. “They’re trying to give so much information, which I think is good.”

Holland teaches history at a small Christian school where she helped make plans for the safe start of the fall semester.

“I had a fairly big role in putting together our policy,” she said. “I wanted to make sure I knew what was happening.”

She also wanted to keep track of state progress to get some idea when her three sons could get back to activities they enjoy.

“I am a mother of three boys who have missed out a ton of life experiences and it’s all because of COVID,” she said.

DHHS has added more pieces of information as the months have passed, but Holland has seen percent positive test calculations change for certain dates, then change back without explanation. It’s difficult to get a complete picture of what’s happening by trying to match numbers of positive tests with other information provided, she said.

“You can’t piece it together,” she said. “There are too many different dates and too many different variables.”

Despite the data holes, Holland remains a close reader of the state data. In the days before the error was acknowledged, DHHS was often hours late in meeting its noon self-imposed reporting deadline.

“I look at every chart every day,” she said. “It’s now an obsession of mine. When they were delayed in reporting, it made me so mad.”

Problems around the U.S.

Other states aren’t immune to reporting problems. The COVID-19 Tracking Project, which collects information from every state and five territories, reported that the same week North Carolina announced its overcount, Texas, California, Florida and Washington also had data problems. The COVID-19 tracker gives North Carolina an A-plus for the quality of its data. Forty states and territories have As or A-pluses.

Some states make more information available to the public. Massachusetts, for example, makes the raw data used to build its dashboard available for download.

NPR reported that 14 states make contract tracing information publicly available, and some provide information on where infections are spreading outside nursing homes and jails.

In an email to the N&O, DHHS said it was looking at ways to display contact tracing information on its dashboard.

At the beginning of the pandemic, North Carolina hospitals were asked to enter by hand the numbers of COVID-19 patients they were treating, how many ventilators were in use, and other information. The system resulted in significant gaps, according to the NC Watchdog Reporting Network.

A months-long effort to automate the survey has most of the state’s biggest hospitals sending information automatically, Charles Carter, chief operating officer for technology and operations, said in an interview this month.

The state reports each day how many people with COVID-19 were hospitalized the day before, but it’s still based on incomplete hospital reporting, and there’s no way for the public to know which hospitals’ information is missing.

Usually, DHHS has information from 80% to 90% of the state’s 112 hospitals. Some of the state’s smaller hospitals aren’t able to use the automated process, Carter said. The state started automating the hospital COVID-19 survey in mid-April, and now about 80 hospitals use it with another 20 working toward it.

This story was originally published August 31, 2020 at 2:52 PM.