AI Expert Answers Prompt Engineering Questions From Twitter

I'm prompt engineer Michael Taylor.

This is Prompt Engineering Support.

[funky upbeat music]

@Marmotses wants to know, serious question,

what is a prompt engineer/prompt engineering?

One of the main things I'm doing every day

as a prompt engineer is AB testing

lots of different variations of prompts.

So I might try asking the AI one thing

and then ask it a completely different way

and see which one works best.

So a prompt engineer might be employed by a company

in order to optimize the prompts

that they're using in their AI applications.

@AdamJonesInk is asking,

does anyone else use please and thank you

when communicating with ChatGPT and Perplexity?

I'm hoping that I'll get better responses

or be treated slightly better

if the AI models ever take over.

Specifically saying please and thank you,

there is no evidence that that improves the results,

but being emotional in your prompts, for example,

using all caps does actually improve performance as well.

So if you say for example,

this is very important for my career

and you add that to your prompt,

it will actually do a more diligent job.

It has learned that from reading Reddit posts

and reading social media posts

that when someone says this is very important for my career,

the other people that answer

actually do answer more diligently.

One thing that we saw last winter

was that ChatGPT started to get a little bit lazy

and what someone figured out

was that when it knows that the date is December,

then ChatGPT actually does get lazier

because it's learned from us

that you should work a little bit less in the holidays.

@snufflupaguss is asking,

do you get better results from LLMs

when you prompt it to imagine

that you're an experienced astrophysicist.

Why would you want them to pretend?

Let's do a little experiment here.

Let's write the prompt as an astrophysicist

and then write the same prompt

as a five-year-old and see the difference.

So I've asked it to tell me about quantum mechanics

in two lines as an astrophysicist

and you can see it uses a lot of big words

that a typical astrophysicist would know.

We can then ask it the same thing as a five-year-old.

And now it's explaining quantum mechanics as a magic world

where tiny things like atoms can be in two places at once.

The overriding rule

is that you should be direct and concise.

As an astrophysicist or you are an astrophysicist,

that tends to work better

than adding unnecessary words like imagine.

@veebrandt is looking for any tips

on how to improve my prompts?

Well there are actually thousands

of prompt engineering techniques,

but there's two that get the biggest results

for the least amount of effort.

One is giving direction

and the second one is providing examples.

Say for example, I had this prompt

and I invented a product where it's a pair of shoes

that can fit any foot size.

Now how can I improve that prompt template?

One thing I can do is to give it some direction.

One person who is famous at product naming was Steve Jobs.

You could invoke his name in the prompt template

and you're gonna get product names in that style.

Alternatively, if you prefer Elon Musk's style

of naming companies, you can provide some examples

of the types of names that you really like.

The reason there's two hashtags in front of this

is that this means this is a title.

It really helps ChatGPT get less confused

if you put titles on the different sections of your prompt.

@petemandik is asking serious question about AI artists,

why does the number of fingers on a human hand

seem to be particularly difficult for them?

The difficulty in rendering fingers

is that it's very intricate

and the physics is quite difficult to understand

and these models were pretty small.

They didn't have that many parameters,

so they hadn't really learned how the world works yet.

We also have a really strong eye

for whether fingers are wrong or whether eyes are wrong.

It's something that we look out for as humans.

One thing you might try in a prompt is to say,

make the fingers look good.

That tends to not work either

because everything in a prompt is positively weighted.

If you say don't put a picture of an elephant in the room,

then it will actually introduce a picture of an elephant.

So what you need is a negative prompt.

That's not always available.

For example, it's not currently available in Dall-E,

but it is available in Stable Diffusion.

So we're gonna type in oil painting hanging in a gallery.

We're gonna hit dream.

So what you can see

is that some of them have a big gold frame,

but the one on the right doesn't have a frame

and I actually prefer that.

So how can I get it to remove the frames?

One thing I can do is if I add up the negative prompt here

I can say frames and the negative prompt,

it's gonna remove that

and now we can see that all of the paintings

don't have frames.

@robertodigital is asking,

what is the weirdest response you've gotten from ChatGPT?

So my favorite one is if you ask it,

who is Tom Cruise's mother,

it knows who it is.

It's Mary Lee Pfeiffer.

If you ask it, who is Mary Lee Pfeiffer's famous son,

it doesn't know that Mary Lee Pfeiffer's son is Tom Cruise,

so it'll make something up.

I think the last one that I got was John Travolta.

So the reason why this happens

is there's lots of information on the internet

about Tom Cruise and who his mother is,

but there's not that much information on the internet

about Mary Lee Pfeiffer and who her son is.

Hallucinating is when the AI makes something up that's wrong

and it's really hard to get away from hallucination

because it's part of why these LLMs work.

When you're asking it to be creative,

creativity is really just hallucinating something

that doesn't exist yet.

So you want it to be creative,

but you just don't want it to be creative

with the facts.

@Swartzchild is asking, I'm not an expert in AI,

but if an LLM is trained on biased data,

then won't that bias come through in its responses?

Well you're absolutely correct

because AI are trained on all of the data from the internet

and the internet is full of bias

because it comes from us and humans are biased too,

but it can be pretty hard to correct for those bias

by adding guardrails

because by trying to remove bias in one direction,

you might be adding bias in another direction.

Famous example was when Google added to their prompts

for their AI image generator service,

an instruction that they should always show diverse people

in certain job roles.

What happened was peoples tried to make images

of George Washington

and it would never create a white George Washington.

In trying to do the right thing and solve for one bias,

they actually introduced a different bias

they weren't expecting.

There is a lot of work in the research labs like Anthropic

as a whole safety research team that have figured out,

you know, where is the racist neuron in Claude,

which is their model, you know,

where is the neuron that represents hate speech?

You know, where is the neuron

that represents dangerous activities?

And they've been able to dial down those features.

@carkujon wants to know how much of the conversation context

does ChatGPT actually remember?

If we chatted for a year with information dense messages,

would it be able to refer back to info from a year ago?

So when you open a new chat session with ChatGPT,

it doesn't know anything about you

unless you put something in your settings specifically.

They do have a feature which is a memory feature

that is experimental

and I don't think it's on by default.

So one trick that I tend to use

is I will get all of the context at one thread for a task.

I'll just ask it to summarize

and then I'll take that summary and then start a new thread

and then I've got the summary more condensed information.

It will get less confused

by all of the previous history.

it doesn't need to know about.

@bybrandonwhite, does customizing your settings in ChatGPT

and providing your bio/personal info help better results?

Yes, I find that you get wildly different results

when you put some information in the custom instructions.

You have two fields.

Custom instructions, which is what would you like ChatGPT

to know about you to provide better responses.

And the second box is

how would you like ChatGPT to respond?

I use ChatGPT a lot for programming,

so I tell it what type of languages I'm using,

what type of frameworks.

I give it some preferences

in terms of how I like my code to be written.

The second box is really anything

that you get annoyed about when you're using ChatGPT,

you could put that in the box.

So for example, some people put quit yapping

and then it will give you briefer responses.

@travisdotmedia asks,

what makes a prompt engineer an engineer?

A prompt engineer is designing that system of prompts

that are being used in the application

and making sure that they're safe for deployment,

makes sure that they work

again and again and again reliably.

And that's the same sort of thing

that a civil engineer is doing with a bridge, right?

Like, they're designing the bridge

and they're making sure that when you drive over it,

it's not gonna crash into the river.

@EigenBlade is asking,

do you think we can draw parallels

between large language models and human brains?

LLMs or large language models

are actually based on human biology.

They're what happens

when you try to make artificial neural networks

simulate what our biological neural networks

do in our brain.

So there are a lot of similarities

and a lot of the things

that work in managing humans also work in managing AIs.

So if you've heard of transformer models,

which is what the LLMs are all based on,

the breakthrough there was figuring out

how to make it pay attention

to the right words in a sentence

in order to predict the next token or word in the sentence.

So that was the really big breakthrough

that was made by Google

and then used by OpenAI to create ChatGPT.

@this1optimistic wants to know

what are tokens?

So let's say I started writing the sentence

LeBron James went to the.

What word could come next?

Well, an LLM looks at all the words on the internet

and then calculates the probability

of what the next word might be.

So this is the token Miami.

Which has a 14% chance of coming next

and we have Lakers, which has a 13% chance of coming next.

We also have the word Los,

which is just the beginning of the word Los Angeles.

Here we have the token Cleveland,

which only has a 4% chance of showing up,

but the LLM will sometimes pick this word

and that's where it gets its creativity from.

It's not always picking the highest probability word,

just a word that's quite likely.

The reason they use tokens instead of words

is it's just more efficient.

When you have a token,

which is a little part of a word like Los,

that can be more flexible

and it can be trained to be used in different contexts .

@AdeyemiTestimo4, what is the best LLM in your opinion?

For me it's Claude 3 Opus.

I agree.

Anthropic who makes Claude 3 is doing a great job.

I'm gonna test this against ChatGPT

and then Meta Llama, which is an open source model

and show you the difference in results.

So the prompt we're using is give me a list

of five product names for a shoe that fits any foot size.

We're testing the model's creativity here,

and you can see that we have UniFit Shoes as one idea.

Adaptix Shoes, which is pretty creative

and OneSize Soles, which is my personal favorite.

I'm just gonna copy this prompt to Claude

and with the same prompt, we get different names.

We have Morphfit, Adaptastep, OmniShoe.

That's my new favorite.

Now we're gonna test it on Llama 3,

which is Meta's open source model.

And you can see it comes up with really different names,

FitFlex, SizeSavvy, AdjustaStep, UniversalFit.

It comes with this text at the beginning

and then it's describing each name as well.

That's not what I asked it to do.

Personally, it's subjective,

but I like the Anthropic Claude response best.

@GMonster7000 is asking

what is the simple task an LLM has done

that has changed your life?

For me personally, it's been the programming ability

that I get from using ChatGPT and Anthropic's Claude.

Those models are so good at writing code

and explaining what that code does

that I have really lost my fear of what I can build.

So if we pop over here to Claude,

I've made up a fake product

which alerts you if your baby is choking.

I'm trying to build a landing page for it

because my developers are busy

and it's actually going through

and just writing that code for me.

Say for example, I don't understand

what this section is doing,

I could just copy that and then paste it at the bottom

and say, what does this do?

And it's gonna give me bullet points

on what that specific code is doing step by step.

And that's the way that you learn with programming.

I find that I just never get stuck when I use this.

One of the coolest things, I've done a little automation

or a little life hack that I use every day

is that I set up an email address

that I can email

with any interesting links I've found that day.

'Cause it will send those to AI, summarize them,

and then put them all into a spreadsheet

for me to look at later.

@kannagoldsun is asking, what is prompt chaining?

If you wanted to write a blog post,

you wouldn't get great results

just by asking it to write it all in one step.

What I find works is if you ask it first

to write an outline, do some research,

and then when I'm happy with the outline,

come back and fill in the rest of the article,

you get much better results and they're comprehensive

and fit the full brief

of the article that you wanted to write.

Not only does it make the thought process more observable

because you can see what happened at each step

and which steps failed, but also the LLM gets less confused

because you don't have a huge prompt

with lots of different conflicting instructions

that it has to try and follow.

@AutomationAce_ is asking how can you automate AI?

That's what's called an autonomous agent

where it's running in a loop and it keeps prompting itself

and correcting its work

until it finally achieves the higher level goal.

Microsoft AutoGen is a framework for autonomous agents

an open source framework

that anyone could try if you know how to code.

And I think that's really the big difference

between ChatGPT, this helpful assistant

that we are using day to day

versus having an AI employee in your Slack

that you can just say, make me more money for the company

and it will go and try different things

until something works.

@mrdrozdov is asking,

how would you prompt the LLM to improve the prompt?

There's actually been a lot

of really interesting research here,

techniques like the automatic prompt engineer technique

where the LLM will write prompts for other LLMs

and this works really well.

I actually use it all of the time to optimize my prompts.

Just because you're a prompt engineer

doesn't necessarily mean you're immune from your work

being automated as well.

@BahouPrompts is asking how long until prompt engineering

or a future field related to this becomes a degree?

Will it be a standalone field

or part of every field being taught?

That's a really great question

because some people I talk to

say that prompt engineering

isn't going to be a job in five years.

It's not even gonna be something we practice

because these models are gonna be so good,

we won't need to prompt them.

I tend to disagree

because of course humans are already pretty intelligent

and we need prompting.

We have a HR team, we have a legal team, we have management.

So I think that the practice

of prompt engineering will always be a skill that you need

to do your job, but I don't necessarily think

that in five years

we'll be calling ourselves prompt engineers.

So those are all the questions for today.

Thanks for watching Prompt Engineering Support.