Coherence and errors in analysis
Posted: Sat May 24, 2025 6:41 am
All systems create pieces of code to help them analyze canada phone number list the data. The process is similar: they create the code (Claude with Javascript and ChatGPT, and Gemini with Python), apply it to the data, and obtain a result from which to draw conclusions. In this exercise, both often make mistakes and correct themselves. Each one errs in different ways:
ChatGPT can get into loops that it doesn't resolve, eventually failing to validate the analysis after several attempts.
Claude usually completes the analysis successfully, but his message size limits cause him to fail again when he needs several attempts, rendering the entire exercise null and void, and having to start over.
Gemini may decide on its own to delete rows that cause errors or leave you with empty data because it didn't transform the format properly and didn't realize it. You correct it and it fixes it, but without manual review, it can play tricks on you.
ChatGPT is more aware of what it's done in previous steps, and Claude doesn't seem to reuse its own analysis and errors in the future. Gemini follows the path without any problem in the data trace, but it's much more lost in following the thread of the conversation than the other two. Scores:
ChatGPT: 4
Claude: 3
Gemini: 3
Total score for Data Manipulation and Correction
ChatGPT: 9/10 Claude: 6/10 Gemini: 8/10
3. Baseline analysis capabilities
We evaluate how AI enters the analysis, understands the data, and helps focus the data analysis we ask of it.
3.1 First approximation to the data sent
They're all equally good at understanding what you're looking for; they approach data by giving it context, understand the type of information it contains, and apply it to the reality explained to them. They're capable of putting their know-how into practice and focusing on practical results. Early versions of analysis with ChatGPT suffered from a lack of understanding of some data. They either treated numbers as text or didn't try to understand the meaning of categories and variables. That's a thing of the past, and the context is quite good in both cases. Even better than what many people apply. Although I expected to find different behaviors, I haven't found major differences between the two in this area. Scores:
ChatGPT: 5
Claude: 5
Gemini: 5
3.2 Facilities they provide us and our own ideas
AIs are capable of proposing KPIs and generating new calculated data. Claude does tend to create metrics and KPIs more related to the context and the case being reported. ChatGPT repeatedly goes over market topics and repeats itself a bit (but still does a great job). Gemini may not have the mathematical expressions Claude uses to express himself, but he usually adds a touch of reality. He tells you how you could use that data or directly visualizes it (without you asking) so you get an idea of its usefulness. Example: ChatGPT proposing
ChatGPT can get into loops that it doesn't resolve, eventually failing to validate the analysis after several attempts.
Claude usually completes the analysis successfully, but his message size limits cause him to fail again when he needs several attempts, rendering the entire exercise null and void, and having to start over.
Gemini may decide on its own to delete rows that cause errors or leave you with empty data because it didn't transform the format properly and didn't realize it. You correct it and it fixes it, but without manual review, it can play tricks on you.
ChatGPT is more aware of what it's done in previous steps, and Claude doesn't seem to reuse its own analysis and errors in the future. Gemini follows the path without any problem in the data trace, but it's much more lost in following the thread of the conversation than the other two. Scores:
ChatGPT: 4
Claude: 3
Gemini: 3
Total score for Data Manipulation and Correction
ChatGPT: 9/10 Claude: 6/10 Gemini: 8/10
3. Baseline analysis capabilities
We evaluate how AI enters the analysis, understands the data, and helps focus the data analysis we ask of it.
3.1 First approximation to the data sent
They're all equally good at understanding what you're looking for; they approach data by giving it context, understand the type of information it contains, and apply it to the reality explained to them. They're capable of putting their know-how into practice and focusing on practical results. Early versions of analysis with ChatGPT suffered from a lack of understanding of some data. They either treated numbers as text or didn't try to understand the meaning of categories and variables. That's a thing of the past, and the context is quite good in both cases. Even better than what many people apply. Although I expected to find different behaviors, I haven't found major differences between the two in this area. Scores:
ChatGPT: 5
Claude: 5
Gemini: 5
3.2 Facilities they provide us and our own ideas
AIs are capable of proposing KPIs and generating new calculated data. Claude does tend to create metrics and KPIs more related to the context and the case being reported. ChatGPT repeatedly goes over market topics and repeats itself a bit (but still does a great job). Gemini may not have the mathematical expressions Claude uses to express himself, but he usually adds a touch of reality. He tells you how you could use that data or directly visualizes it (without you asking) so you get an idea of its usefulness. Example: ChatGPT proposing