I'm not sure I have a great answer for all of your individual questions there but I think I do have a good answer for the very first one.
Try to use it before it is ever written. By that I mean, write unit tests for the code as if it really did exist. Write some of the code that will be using the API before you've even written one line of the API. As you try to use it you'll quickly see what works and what doesn't work in the design you had in mind and you'll be quick to change it to match its actual use because you haven't yet written any of the actual code.
There's never any friction to changing something if you haven't committed any of it to code, but the moment you do, there is often some measure of reluctance to do so.
Apress - Practical API Design - Confessions of a Java Architect - 2008
Pragmatic - Interface Oriented Design - 2006
Read Effective Java by Josh Bloch. The book is great for any Java programmer, but also addresses lots of issues surrounding the creation of a useful API.
You can implement the following guidelines:
1) use established or known methods, examples:
createPatientRecord();
createPatientAppointment();
createPatientCheckup();
2) returning values:
3) consistent arguments ordering/sequence
recallPatientRecord(long patientId, long hospitalId);
recallPatientRecord(long patientId);
recallPatientRecord(long patientId, long hospitalId, int disciplineCode);
This question is almost impossible to answer. It may not even be a proper question for Stack Overflow (Person A: How do I do solve specific problem? Person B: Here is this definitive answer).
1.What are the best practices and patterns to be followed for designing APIs
What programming language are we talking about? What field of problem domains are we talking about? Because what works for one programming language / problem domain may not work well in another programming language / problem domain.
2.How to achieve implementation hiding the best way (C++/Java)
There is an answer to that question, and it is too long to write here. In fact, it is so long that a website reference wouldn't be enough. There are many websites and books, when combined together, will answer that question. Oh, and I'm just talking about C++. Repeat that whole process for Java. And for C#, and for Python, and for .... (etc.)
4.Any reference books/links which guide with neat examples to beginners
Pick a programming language. Then I can provide references to books and links. As Fred Brooks wrote, there is no silver bullet. There is no single mentality that you can take from programming language to programming language. What makes a good API in one programming language would be a design mistake in another programming language. Trust me, I learned this the hard why by trying to apply C++, Delphi, and Java idioms across those languages. It lead to bad APIs and bad code.
There are also competing schools of thought in designing APIs. And, unfortunately, the reality is that nobody is 100% right. In fact, you become a great API designer when you know multiple schools of thought and know when to apply a particular school of thought to a particular problem.
I have a tip regarding point 3 (generic API design):
Start of basing your API on specific use-cases; Make your design specific, not generic - Then genericise later if you discover that the API can be re-used.
In the past I've seen APIs refactored to the point where they are so generic that one of the method parameters is a "Parameters" object or worse still, a DOM tree corresponding to an artbitrary piece of XML; e.g.
void processData(Parameters reportParams);
With this uber-generic approach: