First Arabic AI fact-checking tool

Ta'keed is the first generative fact-checking system created for Arabic claims. It goes beyond simply labeling a claim as true or false. Instead, Ta'keed provides explanations about why a claim is considered credible or not. The system works in two steps. First, it finds relevant information. Then, it uses a Large Language Model, a type of AI, to check the truthfulness of the claim based on the information it found.

Benefits

Ta'keed helps users understand the reasoning behind fact checks. It provides explanations that make it easier to trust the results. This system aims to improve the accuracy and transparency of fact-checking for Arabic content.

Use Cases

This tool can be used by journalists, researchers, and anyone who needs to verify the accuracy of Arabic claims. It is particularly useful for identifying misinformation and understanding the evidence supporting or refuting a statement.

Vibes

Initial evaluations show that Ta'keed is effective. The system achieved an F1 score of 0.72 for classifying claims. When comparing its generated explanations to expert explanations, the system showed a high degree of semantic similarity, with an average cosine similarity score of 0.76. Using the top seven retrieved information snippets led to the best performance, resulting in an F1 score of 0.77.

Additional Information

The creators of Ta'keed also developed ArFactEx, a new dataset specifically for Arabic claims. This dataset includes manually verified references to help evaluate fact-checking systems. The research also explored how the amount of retrieved information affects accuracy, finding that a specific number of snippets improved the system's performance.