Kampfers said:
Mr President Elect said:
Hmm, sounds like your prompt engineering game is a bit weak. Sure, a lot of times the essays seem very middle schoolish, but you can definitely guide it to dive deeper into the specifics and get it to a pretty high quality. But r yeah, there are times that it will very confidently give you wrong answers, so got to be mindful of that and verify the details.
I played around quite a bit with different prompts, refining the prompt in subsequent attempts, etc. It's not at that level yet.
There's a significant difference between what it does (which is still very impressive) and original thought. Any upper-level college course (and certainly any PhD program) will require works of original scholarship. That means that you can't just regurgitate what is out there. You have to take what is out there and shape/craft a unique argument. The bot is a long way from being able to do that. Even if I give it the "takeaway" that I want the paper to have - it struggles to connect that to the broader topic in anything but the most general/descriptive language.
Don't get me wrong, the results are still impressive. And they might be enough to convince a layperson that the bot (or a person reading from the bot's script) knew what they were talking about. But if you are actually involved in the field - the bot isn't there yet.
It's use of quotes is haphazard - it struggles to identify what the most relevant items to quote are and whether those quotes are truly in line with the argument it is putting forward. I asked it to include data - it included some data, but hardly gave enough context to make that data's inclusion worthwhile. It seems to know that it should use quotes/data when asked, but fundamentally doesn't understand how to incorporate those into its writing. It also does only the most fundamental quote blending - "So and so said 'blah blah blah.'" I doubt it is capable of more - prompting it to only rewrite/blend the quotations doesn't produce any results of note.
I have yet to see it put together a paragraph with more than three sentences in it. It makes fundamental mistakes from time to time - such as identifying a court case that was decided by the European Court of Human Rights as being a case that was decided by a Constitutional Court of a member state of the EU. That's in addition to the times when it is "confidently wrong" in the conclusions it makes, as you noted.
Do I think you could use this to summarize a lot of things and provide a high-level overview? Sure. This could generate Wikipedia articles in a flash (with the drawback that the articles wouldn't have any cited sources). Could you use this in any sort of serious academic writing? No.
Your are kind of moving the goal post. Your original quote:
Quote:
but it showed a level of research and comprehension of the topic that shouldn't earn a high school student a passing grade
I agree that we are still a ways away from having it include original thought into the essay's. That is more AGI which would also include the ability to extend modern physics and math. With that being said, you could still have this inject original thought via prompts and connect it with the broader subject. Do I think this could write a PhD paper, no way. Do I think it could significantly aide the writing of one, absolutely. Do I think it could write a lower-level college essay? Absolutely, while not blindly but with prompts and just a little bit of injection.
Also, you can simply tell it to make body paragraphs roughly 5 sentences long.
But just with the small amount that I have played with it (and mostly for programming), if I wanted it to write an essay for me, I would do the following:
prompt 1 abbreviated: Give me an outline for an essay on blah blah, with give 20 points to support the argument.
prompt 2 abbreviated: Write a 2,000 word essay using points 2, 4 and 7 from the outline above. In the body paragraph using point 4, combine it with point 12 from the outline.
prompt 3 abbreviated: In body paragraph 3, can you write it from the perspective of someone who has the opinion that "blah blah blah"
prompt 4 abbreviated: Can you rewrite the essay in the writing style of J. K. Rowling
Now, I am just sh*tposting here, but I imagine that after spending the 5 minutes it took to the above, you will have a pretty decent essay that might need a little tweaking. As an engineering student that hated writing papers, this would have been a godsend.