In this research, we aimed to determine whether students' math achievements improved using ChatGPT, one of the chatbot tools, to prepare lesson plans in primary school math courses. The research was conducted with a pretest-posttest control group experimental design. The study comprises 39 third-grade students (experimental group = 24, control group = 15). The implementation process lasted five weeks and 25 lesson hours. In the experimental group, lessons were taught according to plans prepared using ChatGPT, while in the control group, existing lesson plans were used. Students' academic achievement was measured with a multiple-choice achievement test of 25 questions with two separate questions for each learning objective. According to the results, students' academic achievement increased significantly (d = 1.268) in math lessons taught according to lesson plans prepared using the ChatGPT. Although there was a difference between the posttest scores of the experimental group and the control group in favor of the experimental group, it was determined that this difference was not significant. These results show that teaching primary school math according to lesson plans prepared using ChatGPT is effective in academic achievement. Teachers should consider ChatGPT and their plans, combining them and benefiting from both in the implementation process.