I try to make the uipanel change boarder colors while pressing and releasing mouse button on elsewhere except inputs and panel buttons.
function [oldpropvalu
You did not show how you produced the error, but from the error message, I guess you called the function with less than 4 input arguments. Then varargin does not exist, so matlab gives the error.
To avoid the error, you need to check nargin before you use varargin, for example, replace your if statement line with
if nargin==5 % so length(varargin)==2