What in particular makes it seem more difficult to you? There are some raags with simple arohanas and more complicated avarohanas. Can't remember the one I was looking at yesterday but the avarohana was at least twice as long!
But if you're looking at a straight, symmetrical aroha/avaroha pattern (say Bhupali...SRGPDS', S'DPGRS) and it is still more difficult, it could be that just shifting up the neck feels stranger, rather than coming down it. Also, in the ascent, using the index finger, you have the second finger there to stretch and hit the final note, where when descending (in a straight pattern) its all index finger.
Such is my limited experience so far. Like most things, practice will help, over time!