I want to use 3rd party authentication (OpenID, maybe OAuth but I guess OAuth is meant for authorization) so that user can login easily.
But does authenticatin
I believe both the answers by Renat Gilmanov and flup should be useful to you, but I will try to just answer the question that was asked, here.
No, it does not mean you contact the 3rd party site on every request. In fact, you can't, since the whole OpenID process should just happen once for a given session (it's a somewhat annoying manual step for the user)
I'll refer to the OpenID provider (3rd party) as Google, since that's the example you gave in the question. The only thing that Authentication will do for you is give you Google's assurance that the person making that particular request also knows the password to the Google account name they gave you.
After that, it's up to you to keep track of requests coming from the same "person" and treating them as the same account. Basically, however you're handling the rest of the user's "session" information, you can now assume that all requests in this session are from that user.
The most common way would be through passing a cookie to the browser immediately, which contains some identifier that you keep track of on the server, and that confirms for you that whoever passes you that cookie is also the person who knew the password for that Google account. Another option would be sending custom HTTP headers, which is preferable in certain ways, but trickier to do by hand.
You could build all of this manually, but I would very strongly recommend finding some library code to take care of as much of this for you as possible. You don't mention what you're using to build this Web Application (in fact, you don't explicitly say it's a web application, I just gleaned that from the way the question is tagged), but there are plenty of choices for pretty much every framework or system you're likely to be using.
On some level, everything I wrote applies if you're writing an API for a native mobile app to connect to, as well. But you may have to jump through some slightly more complicated hoops in order to have the user authenticate, since some OpenID providers assume you're coming through a web browser.