When to besiege?
If you see a fief with 0 loyalty, no expenditures and no keep levels,. take it; it's a piece of cake. As the loyalties go up and the keep level increases and the garrison expenditures go up and the forces there increase, it becomes harder for you to take fiefs. Sieges of fiefs with large keeps usually require large armies; 5,000 to 10,000 men to be consistently successful. But most fiefs can be taken with smaller armies, although with more difficulty. Remember, in the beginning of the game, most French fiefs have low keep levels, and English fiefs (especially those in France) have high ones. As the game goes on, all players, and especially the French, build up their keep levels.
Also remember to check the current ownership of a fief before you start a siege. Another of your allies may have already taken the "enemy" fief and would not appreciate the damage you would inflict while taking it again.