I have a set S of points (2D : defined by x and y) and I want to find P, the smallest (meaning : with the smallest number of points) polygon enclosing all the points of the
You probably mean you want the smallest area, which is the convex hull.
If you really do want the fewest points, you can just make a triangle with super-large vertex positions so that all your points are enclosed.