Well there isn't really any adjustment. Both out side and inside handles are performing the same task, and moving a lever, or pushing one to operate the mechanism. If you pop off the door panel, *possibly * your rod is in a bind or installed improperly, creating a situation where it takes more force to open the door.
I would start there.
The only other thought is the door may be slightly out of alignment and dragging, so from the inside your leverage is upon opening the door is a little different vs. the outside where the handle is located and you are standing it may seem easier?
Good luck, let us know what solves the problem.