Which is better for coding: ChatGPT vs Bard?

46

ChatGPT vs Bard: For programmers, generative AI offers tangible benefits. It helps with writing and debugging code, making our busy lives a little easier. But now there are competing tools like ChatGPT and Bard, which begs the question: which one is best for me?

We’re pitting these tools against each other in the ultimate battle to see which tool is the richest for programming purposes right now.

ChatGPT vs Bard: What’s the difference?

The biggest difference between ChatGPT vs Bard is the large language models (LLM) on which they are powered. ChatGPT uses Generative Pre-trained Transformer 4 (GPT-4), while Bard uses the Language Model for Dialogue Applications (LaMBDA). ChatGPT is also developed by OpenAI, while Bard was created by Google.

In terms of abilities, they both do very similar things. Developers can use ChatGPT to:

• Suggestions: Both models can suggest the correct syntax and parameters for functions and other code constructs.
• Completion: Can complete the code you started writing.
• Debugging: Can help you identify errors and problems in your code.
• Explanation: Can explain the code it generates or the code you enter.

Both models were trained on a large dataset, including Common Crawl, Wikipedia, books, articles, documents, and content downloaded from the Internet. However, Bard is a bit different in that it was trained on conversations and dialogs from the web, while ChatGPT was trained mostly on scraped general content.

Both products are still in development, Bard slightly more so than ChatGPT. But to really show how these differences really matter in a practical sense, here’s how they perform when tested against each other.

Testing ChatGPT and Bard against each other

We selected seven categories to test chatbots against each other based on what we thought were most relevant: code generation, troubleshooting, code refactoring, debugging assistance, third-party UI plugins/extensions, ease of use, and cost. Let’s say up front that this is not a strictly scientific comparison, but rather a few tests mixed with our practical experience so far.

1. Code generation

Bread and butter, what do we want it for, right? To this end, we presented the following challenge: “Write a function in python that returns a list of 100 prime numbers.”

Bard’s attempt at coding

def get_prime_numbers(n):
  """Returns a list of prime numbers up to n."""
  primes = []
  for i in range(2, n + 1):
    is_prime = True
    for j in range(2, i):
      if i % j == 0:
        is_prime = False
        break
    if is_prime:
      primes.append(i)
  return primes


def main():
  """Prints a list of 100 prime numbers."""
  primes = get_prime_numbers(100)
  print(primes)


if __name__ == "__main__":
  main()

It looks solid, but does it work? In our experience, chatbots can sometimes create code that won’t run, even going so far as to invent functions! This is one of the reasons why having AI generated code in production without human oversight would be very bad. So let’s try it and see.

Attempting to encode ChatGPT

def generate_primes(n):
    primes = []
    sieve = [True] * (n + 1)
    for current in range(2, n + 1):
        if sieve[current] != 0:
            primes.append(current)
            for multiple in range(current, n + 1, current):
                sieve[multiple] = False
    return primes

# Generate first 100 prime numbers
# The 100th prime number is 541, so we set the limit to 550 to make sure we get at least 100 primes
primes_100 = generate_primes(550)[:100]
print(primes_100)

When we run the same prompt using ChatGPT, this code defines a function generation_primes(n) that takes an integer n as input and returns a list of all primes up to n. It uses the Sieve of Eratosthenes algorithm to generate primes.

Both produced a functional result. The ChatGPT example is better in this case because it is cleaner and has much less code. It also correctly interpreted our vague challenge. But this is just one use case: chatbots are known to vary a lot in their responses depending on the language they’re using (less common or documented languages ​​have more errors) or the complexity of the query. So let’s see how Bard and ChatGPT handle different use cases.

Winner: ChatGPT 🏆

2. Problem Solving

Let’s see how these LLMs solve the problem. Instead of just asking them to write a generic function, we throw at them a programming problem from Leetcode:

Write JavaScript code that enhances all arrays so that you can call the array.last() method on any array and it will return the last element. It should return -1 if the array contains no elements.

ChatGPT problem resolution

Here's how you would do it in JavaScript:

Array.prototype.last = function() {
  if (this.length === 0) {
    return -1;
  } else {
    return this[this.length - 1];
  }
};

In this code:

Array.prototype.last adds a new method called last to the Array.prototype object. This ensures that all arrays will have this method.

this refers to the array object on which the last method is being called.
this.length gives the number of elements in the array.
If the array is empty (this.length === 0), the method returns -1.
Otherwise, the method returns the last element of the array: this[this.length - 1].

Here is how you would use it:

let array = [1, 2, 3, 4, 5];
console.log(array.last()); // prints: 5

array = [];
console.log(array.last()); // prints: -1

Remember that extending built-in JavaScript objects like Array is generally not recommended, as it can lead to conflicts if the JavaScript environment (like a browser or Node.js) adds a method with the same name. This is generally called monkey patching and is usually avoided in production-grade code. But for quick and dirty hacks, or to understand how JavaScript works, it's a fun tool to use!

ChatGPT doesn’t just solve the problem, it explains the problem. Elegant! Let’s run leetcode through the code and see if it actually passes the tests.

Looks like ChatGPT fixed the problem on the first try! Now it’s time for the Bard to step up to the plate.

Bard’s solution to the problem

Sure, here is the JavaScript code that enhances all arrays such that you can call the array.last() method on any array and it will return the last element. If there are no elements in the array, it should return -1:


// Define a new method on the Array prototype
Array.prototype.last = function() {
  // If the array is empty, return -1
  if (this.length === 0) {
    return -1;
  }

  // Otherwise, return the last element
  return this[this.length - 1];
};


Here is an example of how to use the array.last() method:


const array = [1, 2, 3, 4, 5];

// Get the last element of the array
const lastElement = array.last();

// Print the last element of the array
console.log(lastElement); // 5

Okay, so we didn’t get a very thorough explanation of how the code works from Bard. However, it is almost identical when you compare what ChatGPT and Bard have created. Did they watch each other’s homework?

Naturally, we can’t give Bard a word, so we run the test through Leetcode and…

… It works! Which was to be expected since the answer was almost identical to ChatGPT’s answer.

When it comes to solving problems, it’s hard between them. Although they produce almost identical results, ChatGPT does a much better job of explaining the code to you, which is pretty good for actually learning how to fish instead of ChatGPT just fishing for you. This is our overall experience with ChatGPT: not only do you get a possible solution to your problem, but ChatGPT walks you through it a little more.

Winner: ChatGPT 🏆