Hmm...
There have been few games I have played with moments which scared me but King's Field IV (PS2) scared me for the most of it, imagine a first person Resident Evil with slower graphics and set in Medieval times. Scary.
The other is from another relatively unknown game, Orphen: Scion of Sorcery (PS2). 3 moments.
I. During one part, you have to go through a maze with parts which block some of your paths when they see you. It wouldn't be too scary, but they have big eyes on them...
II. Another is a room with spiked traps. There are 5 of them, touching one twice kills you, you have to jump over them or go under them, but there are parts of the floor with traps which could make them move.
III. A boss, which runs at you and pushes you down a hall, where eventually you'll be pushed of the edge, I shake really badly versing it, even now.
I got the game when I was about 8, now, 6 years later they STILL freak me out.