Inversion of return value

Inversion of return value

I have a flag that I want to pass to a function which returns true or
false based on a value in a map:
// userList is a List<String> and is stored as the value field in a map
// user is a String
if(flag)
{
if (userList == null)
return false;
else if(userList.size() == 0)
return true;
return userList.contains(user);
}
else
{
if (userList == null)
return true;
else if(userList.size() == 0)
return false;
return !userList.contains(user);
}
My question is this: is there anyway to tidy this code up, there is a lot
of replication (the if and else block are identical, except their return
values are the opposite of each other).
I'm not a very experienced code, and I'd really appreciate some guidance!