We present an immersed boundary (IB) method for solving fluid flow problems in presence of static and moving rigid objects. A finite element method is used starting from a base mesh which does not represent exactly rigid objects (non body-conforming mesh). At each time step, the base mesh is locally modified to provide a new mesh fitting the boundary of the rigid objects. The mesh is also locally improved using edge swapping to enhance the quality of the elements. The Navier-Stokes equations are then solved on this new mesh. The velocity of moving objects is imposed through standard Dirichlet boundary conditions. We consider a number of test problems and compare the numerical solutions with those obtained on classical body-fitted meshes whenever possible.