Generative AI: Limitations
Inaccuracies
ChatGPT has been known to generate content that is factually wrong. It also has difficulty with math and logic. It is known to “hallucinate” – which means to create plausible-sounding responses that are incorrect and even nonsensical. It is likely that accuracy will improve over time.
Bias
Content generated via ChatGPT and media creators such as Midjourney will reflect the information it was trained on. Information scraped from the internet is full of stereotypes, misinformation, and biased data. While fine tuning of data is reducing the overt presence of bias in results over time, it is deeply embedded.
Privacy and Security
ChatGPT has experienced security and privacy breaches related to user data. While issues are being addressed, the newness of the tool suggests there could be unintended impacts. There is also concern related to the use of user data within the dataset that trained ChatGPT. However, GenAI tools, such as ChatGPT, have data control settings that allow individuals and companies to control their chat history and what gets added to the training data.
ChatGPT Data Controls FAQ
Intellectual Property
The text and art created by generative AI tools is based on works of individuals scraped by the Internet and made part of the training dataset. Therefore, ownership of this content is murky. In the US, various lawsuits are underway that accuse GenAI companies of copyright infringement. Furthermore, companies like OpenAI, who owns ChatGPT, are entering paid licensing agreements with news agencies for the use of their content. Copyright protection in Canada of content created via the use of GenAI tools is also not clear. For further insight, consult the University of Toronto’s Generative AI Tools and Copyright Considerations article.
Inequity
A gap may develop between who can and cannot access AI tools like ChatGPT – depending on how these tools are funded and integrated into existing platforms. For example, ChatGPT Plus, which uses the more sophisticated ChatGPT 4, is only accessible via a paid monthly subscription. Microsoft currently integrates ChatGPT 4 into their Bing Chat (now branded as Bing Copilot) which has become part of their free Bing search feature.
Adapted from AI and Teaching at Duke, Learning Innovation, Duke University, and 8 Problems With OpenAI’s ChatGPT.