I have an (OCT) image like shown below (original). As you can see, it mainly has 2 layers. I want to produce an image (shown in the 3rd picture), in which the red line indic
The following set of instructions (using Matlab Image processing toolbox) seems to work well for your image:
Blur your image (Im) with a median filter to decrease the noise:
ImB=medfilt2(Im,[20 20]);
Find the edges using sobel mask and dilate them a little bit to connect close components, and 'clean' the overall image to get rid of small areas
edges = edge(ImB,'sobel');
se = strel('disk',1);
EnhancedEdges = imdilate(edges, se);
EdgeClean = bwareaopen(EnhancedEdges,1e3);

You then have your two zones, that you can detect separately using bwlabel
L=bwlabel(EdgeClean);
Finally, plot the two zones corresponding to L=1 and L=2
[x1 y1] = find(L==1);
[x2 y2] = find(L==2);
plot(y1,x1,'g',y2,x2,'r')

There are not a lot of steps because your initial image is quite nice except from the noise. You may need to play a bit on the parameters as I started from a downloaded version of your image which can be of lesser quality than the original one. Of course this is a minimal code, but I still hope this will help.