A computer program ranking the check-worthiness of claims made by politicians continues its advancement this election season. Check-worthiness describes whether or not a claim should be fact-checked.
The Claimbuster program provides users with new features, like testing the check-worthiness of statements in text entries, and it now includes information from both parliamentary meetings and presidential debates.
What it is
Claimbuster scores statements on a scale from zero to one. The lower something is scored, the less check-worthy it’s considered. If a sentence is deemed more check-worthy, it’s ranked at a higher number.
The Claimbuster team took transcripts from presidential debates beginning with John F. Kennedy versus Richard Nixon, and went sentence by sentence, ranking if there was an important, unimportant or no factual claim.
All the information was introduced into and examined by the program, said Mark Tremayne, co-principle investigator and communication assistant professor.
The computer then learns what a sentence that expresses a factual claim looks like, Tremayne said, a process called 'machine learning.'
There’s been a pretty strong correlation between claims Claimbuster scores higher and those fact checked by PolitiFact or CNN, Tremayne said. Claims scored low by Claimbuster are rarely selected by professional news organizations themselves.
“This is a tool that journalists could use or other people could use to help them in their fact-checking,” he said. “It can basically give you, in rank order, the sentences you want to look at.”
Claimbuster has been ranking claims made at the recent presidential debates live, Tremayne said. Chengkai Li, principle investigator and computer science and engineering associate professor, took a converter box designed to translate broadcast signals into closed captions and connected it to the computer to assist with the process.
It’s a bit choppy, he said, but it gives better results than expected.
“As the debate’s going, we’ve got sentences popping up on our screen,” Tremayne said.
Claimbuster has also been ranking the claims of the Parliament of Australia. The website has record of meetings since Feb. 2 of this year, broken down by topics discussed.
Users can also insert any type of text or Tweets to Claimbuster’s website, which then ranks on the check-worthiness of the information.
The next step
The next step is actually fact-checking the claims, Tremayne said.
One idea is coming up with a database consisting of old fact checks that have already been verified that the program could pull from and use, he said.
Alexios Mantzarlis, director of the International Fact-Checking Network at The Poynter Institute, said he doesn’t think automated fact-checking will ever be 100 percent automated, but with the speed of social media, it needs to get quicker.
Automatic tools are going to impact journalism and different job fields in the future, Tremayne said.
Fact-checking in the presidential election
Political science senior Stephen Perkins said he attributes a greater interest in fact-checking to the two current presidential candidates, Republican nominee Donald Trump and Democratic nominee Hillary Clinton.
People tend to trust these two candidates less than past candidates, he said.
Mantzarlis said interest in fact-checking rose in the 2008, 2012 and 2016 elections. This year is heightened in part because of the candidates. One’s been brazen, he said, while the other has been more guarded.
The media and fact-checking
The interest in fact-checking has risen over time. Tremayne said journalism in the '80s and '90s was more “he said, she said," and less interested in getting to the truth. PolitiFact, an online initiative which researches the truthfulness of statements, served as a trailblazer for fact-checking organizations, he said.
The internet has played a part in the evolution of fact-checking, Tremayne said. Information is easier to access now through search engines like Google.
The media has a dangerous task, Tremayne said. When people select claims to fact-check, there’s potential for others to call bias. The benefit of having software that flags claims from both candidates is that it could eliminate that potential, he said.