i am using SwipeRefreshLayout in my below layout:
Just create a class which extends SwipeRefreshLayout and override the method canChildScrollUp(). Return true when you want scroll down for your control.
For example for scrollview you may try this,
@override.
boolean canChildScrollUp()
{
//your condition to check scrollview reached at top while scrolling
if(scrollview.getScrollY() == 0.0)
return true;
else
return false;
}