In interactive e-learning environments such as Intelligent Tutoring Systems, there are pedagogical decisions to make at two main levels of granularity: whole problems and single steps. Here, we focus on making the problem-level decisions of worked example (WE) vs. problem solving (PS) and the step-level decisions of elicit vs. tell. More specifically, we first investigate the impact of decision granularity on student learning and then explore taking granularity into account in data-driven pedagogical policy induction. In a series of classroom studies, we explored the impact of three types of granularity: problem-level only (Prob-Only), step-level only (Step-Only), and both problem and step levels (Both) on student learning. Results showed that Prob-Only can be more effective for Low-incoming competence students, Step-Only can be more effective for High ones, and Both can be effective for both Low and High students. This suggests that granularity indeed can have an impact on student learning. However, there was no significant difference among the three granularity conditions overall. One possible reason is that the pedagogical decisions were randomly made rather than adaptively. Prior research has shown that effective pedagogical decision-making can significantly improve student learning. In recent years, there has been growing interest in applying data-driven approaches to induce pedagogical policies directly from student-system interaction logs. Though, most of the prior works treated all system decisions "equally, or independently" without considering the long-term impact of higher-level actions or the interaction of decisions made at different levels. Here, we apply reinforcement learning (RL) to induce pedagogical policies that make decisions at different granularity levels and evaluate their effectiveness in empirical classroom studies. We first applied RL to induce a problem-level and a step-level policy and evaluated their effectiveness in a classroom study by comparing them with two random "yet reasonable" policies, one at the problem-level and one at the step-level. Results showed that there was no significant difference between the two RL conditions, and none of them was significantly more effective than the two random baseline conditions. The results suggest that RL induced policies that make decisions at a single granularity level may not always be effective. On the other hand, results from the granularity studies showed that Both-level decisions can benefit more students than either the problem- or step-level, and thus, considering both levels of decisions in RL policy induction may lead to effective policies. Therefore, we then applied an offline, off-policy Gaussian Processes based Hierarchical Reinforcement Learning (HRL) approach to induce a hierarchical pedagogical policy that makes decisions at both the problem and step levels. In an empirical classroom study, the HRL policy was compared with a Deep Q-Network (DQN) induced step-level policy and a random yet reasonable step-level baseline policy. Results showed that the HRL policy was significantly more effective than the DQN induced policy and the random baseline policy. The results suggest that by taking decision granularity into account, HRL indeed can induce effective policies that can significantly improve student learning. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://bibliotheek.ehb.be:2222/en-US/products/dissertations/individuals.shtml.]