Model predictive control is a widely used optimal control method for robot path planning and obstacle avoidance. This control method, however, requires a system model to optimize control over a finite time horizon and possible trajectories. Certain types of robots, such as soft robots, continuum robots, and transforming robots, can be challenging to model, especially in unstructured or unknown environments. Kinematic-model-free control can overcome these challenges by learning local linear models online. This paper presents a novel perception-based robot motion controller, the kinematic-model-free predictive controller, that is capable of controlling robot manipulators without any prior knowledge of the robot's kinematic structure and dynamic parameters and is able to perform end-effector obstacle avoidance. Simulations and physical experiments were conducted to demonstrate the ability and adaptability of the controller to perform simultaneous target reaching and obstacle avoidance.