It knows how to create sentences that sound like something a human would write, and it's even good at understanding context. But that's it, it has no actual intelligence, it doesn't actually understand the code, and most importantly, it's not able to say "Sorry chief, I don't actually know what this doing, look it up yourself."
I don't want to sound negative of course, and I expect many of these apps coming up, until Codex stops being free (if they put it on the same pricing as text DaVinci model, which Codex is a fine-tuned version of, it will cost a ~cent per query). I'm just wondering how come the information about this type of app reaches most people way before the information about "the existence of Codex" reaches them.
For all the publicity around Codex recently (and especially on HN), it still seems like the general IT audience is completely unaware of the (IMHO) most important thing going on in the field.
And to anyone saying "all these examples are cherrypicked, Codex is stupid", I urge you to try Copilot and try to look at its output with the ~2019 perspective. I find it hard to beileve that anything but amazement is a proper reaction. And still, more people are aware of the recent BTC price, than this.
Source: have been playing with Codex API for better part of every day for the last few weeks. Built an app that generates SQL for a custom schema, and have been using it in my daily work to boost ma productivity as a data scientis/engineer/analyst a lot.
And does having this reduce the amount of discomfort badly readable code creates, and thus make you less inclined to take care the code is and stays easily readable?
i = 0x5f3759df - ( i >> 1 );
/* we don't use the actual price but the discounted price, as per email from Manager Bob on 2022-09-16 */
subtotal += price * customer_discount_factor;
or
/* note there's a 2ms delay while relays settle; this is subtracted from sample time, so timeout is not what you might expect */
select(0,&readfd,NULL,NULL,&timeout);
Actual human documentation would read something like:
> Return true if the X-HELPSCOUT-SIGNATURE request header matches the
> base-64 encoded SHA1 hash of the raw request data.
Probably this still gets confused like humans do when variables and functions are named less clearly or even plainly wrong.
I wonder if reading explanation like this makes you more likely to believe code is correct, even if some details are wrong.
In this signature example, you can read the wrong header, calculate hash the wrong way, compare hashes wrong way, etc. there are some many tiny mistakes.
Wound up being useful for explanations of long code paths across file boundaries in large code bases.
If it could give you some context about the implications I could see it being handy for static analysis one day.
That is, it's like it generates these kinds of comments:
That is, IMO the whole purpose of comments should be to tell you things that aren't readily apparent just by looking at the code, e.g. "this looks wonky but we had to do it specifically to work around a bug in library X".Perhaps could be useful for people learning to program, but otherwise people should learn how to read code as code, not "translate" it to a verbose English sentence in their head.