It really depends on what type of game/application you are trying to create. A simple game then SFML is fne but with something with more advanced physics then you will need Box2D. Try without Box2D and if it goes well, then continue without Box2D but if you need more advanced physics then use Box2D.