Bing Chat is public large-language model (LLM), that is integrated with ChatGPT 4, is hosted by Microsoft, and is best used on the Microsoft Edge browser. The OpenAI and Microsoft relationship is confusing to me, but I thought I would play with Bing Chat anyway. (Here is an article and diagram on their confusing partnership)
Before I started playing with Bing Chat for this blog I looked at the ZDNet article, “What is Bing Chat? Here’s Everything You Need to Know.”
The basics:
- Bing Chat supposedly can do citations
- There are no restrictions on date. If it was online today, it can be found
- Lots of plugins, including OpenTable and Wolfram Alpha, including Kayak, Klarna, Redfin, and Zillow. BTW I have no idea why/how Bing Chat would be in OpenTable etc. But the article did mention travel queries so I would assume that is where Kayak comes into play.
- Bing Chat is also in Windows 11 through a new Windows Copilot integration
- You can use it for images
According Microsoft “the new Bing is faster, more accurate, and “more capable” than ChatGPT or GPT-3.5, the LLM behind ChatGPT.” Bing Chat uses ChatGPT 4.0 which is the only way you can use ChatGPT 4.0 for free.
So I did a little playing around. Please note Bing Chat responses are screenshot images. I found this made it a little easier to show the output style and also more clearly define my output vs Bing Chat’s output.
I decided to test 3 things: Citations, dates (timeliness) and for fun I wanted to test images.
Citations, Timeliness, and Real World Examples:
Prompt: Write 2 paragraphs on the use of ChatGPT in medical libraries and provide exampes of ChatGPT use in medicine with URLs to those examples.
Bing Chat’s response:
Ok it is a little weird to see my blog as the first thing listed “for more insights.” But the links did work when I clicked on all of them.
Let’s try some medical stuff.
Prompt: Find 3 articles and provide citations on exercise induced asthma.
Bing Chat’s response:
Again the links worked when I clicked on them, including the one to UTD. But I was interested in getting some articles in PubMed so I refined my prompt.
Prompt: Search PubMed for 3 citations on exercised induced asthma, provide the PMID and a link to the article.
Bing Chat’s response:
The PMID’s were valid and the links to the articles went to the website where you could get the full text if you had a subscription.
PubMed Results vs Bing Chat:
I was interested in how Bing Chat’s results were different from a quick search on PubMed. Below are 2 screen shots from 2 searches. The first is exercise induced asthma searched as a keyword in PubMed. The second is exercise induced asthma searched as a MeSH term. Both searches were set to display Best Match. Since the articles Bing Chat retrieved clearly weren’t in most recent order, I was curious if Bing would retrieve things within PubMed more closely aligned with Best Match.
I have no idea why Bing Chat chose those 3 citations from PubMed, none of those citations were the most recent nor were they listed in the 50 citations displayed on PubMed’s Best Match results. – Weird
Images:
So what can Bing Chat do with images? I did some playing around with dog images. Note I uploaded an image of my dog, Bear. He is a Mountain Cur mix… so that is a little unfair that I did not upload a picture of a pure bread dog.
Bear with his favorite toy.
Prompt: What breed does this dog resemeble?
Bing Chat’s Response:
Wow that was not even close and the photo below Bing Chat’s response is certainly not a Golden Retriever.
Prompt: Show me a picture of a Mountain Cur mix dog
Bing Chat’s Response:
There we go… more like it. Especially the dog in the middle looks a lot like my pup.
Clearly Bing Chat had difficulty identifying the image I supplied but when I knew what type of image I wanted to retrieve, it did a good job.
Medical Images:
I don’t have any medical pictures that are copyright free AND aren’t already on the Internet where Bing Chat would be able to pull some sort of metadata on. So I didn’t test its capabilities of identifying medical images I supplied.
But I did ask it to retrieve pictures of asthmatic lungs. Yes that is very broad, that was intentional since I wanted to compare Bing Chat’s image retrieval to that of Google Images.
Prompt: Show me pictures of asthmatic lungs.
Bing Chat’s Response:
Google Images:
Obviously Google Images has a lot more images. I think Bing Chat is limited by its output style. However, it was interesting to see that the 3 images Bing Chat chose were not anywhere on the first page of results for Google Images. However, they were on the first page of Bing’s Images page (similar to Google Images). So clearly Bing Chat’s AI favor’s Bing search results over Google’s. (makes sense)
What was really weird and a little concerning was when I clicked on an image from Bing Chat, it was a little difficult to see where the image was from AND Bing Chat launched another tab on my browser to Bing Images and the top part of the page for that tab listed asthma medications that were for sale.
Screen shot: when I clicked on an image from Bing Chat.
Note the list of websites on the right…where is this image from? Not real obvious.
Note behind the giant black square the image is featured on, you can just see pill bottle sticking out on the left, that is the new tab that was opened on my browser.
Screen shot: Here is a screen shot after I closed the Bing Chat image. The newly opened tab is shows asthma medicine for sale above asthma lung images.
It is important to note if you search Bing for images (like you would Google Images) and don’t use Bing Chat to find images, you don’t get drugs listed for sale at the top (as seen in this screen shot below).
Bottom Line:
I think Bing Chat does a better job searching and finding text type of information. The links went to the correct places, I did not detect any hallucinations like I did with ChatGPT 3. I am still not sure why it chose the PubMed citations it did. Why were those better to show than a citation from one of PubMed’s 50 Best Match?
I am still sticking with PubMed for searching. But I am at least happy that it is actually retrieving the correct stuff.
I think there are a lot of concerns regarding it interpreting images, finding images, clearly showing the source of the images, and I am still irritated or unnerved that it opened a new tab on my browser featuring asthma drugs so prominently.
What interests you? What would like to see me try and test and play with next?
Let me know by commenting or emailing me your suggestions.