Problem Solving: A level of performance is being met. Something happens which creates a problem - performance drops. We need to find out what caused the problem and then figure out ways to fix the problem. The objective is to get performance back to where it should be.
For Example: A production line has an established run rate of 1000 books per hour. All of a sudden the run rate drops to 839 books per hour. Something happened, we don't know what problem caused the production rate to drop by 161 books per hour. We need to find out and fix the problem so we can get back to producing 1000 books per hour. Problem solving is focused on closing this performance gap.
Creative Problem Solving: The objective is to find ways to create new levels of performance that we haven't reached before.
For Example: We routinely produce 1000 books per hour on our production line. In this example creative problem solving could be directed at: finding ways to produce 1500 books per hour; finding other ways to use the production line; finding ways to reduce the cost to run the production line, etc.
Lateral Thinking is a method/tool kit that can be applied during the problem solving and creative problem solving processes to help come up with possible workable solutions.
(reference: Lateral Thinking, Edward De Bono)