Long-term memory knowledge is considered to impact short-term maintenance of item information in working memory, as opposed to short-term maintenance of serial order information. Evidence supporting an impact of semantic knowledge on serial order maintenance remains weak. In the present study, we demonstrate that semantic knowledge can impact the processing of serial order information in a robust manner. Experiment 1 manipulated semantic relatedness effect by using semantic categories presented in subgroups of items ("leaf"--"tree"--"branch"--cloud--sky--rain). This semantic grouping manipulation was compared to a temporal grouping manipulation whose impact on the processing of serial order information is well-established. Both the semantic and temporal grouping manipulations constrained the occurrence of serial order errors in a robust manner: when migrating to a nontarget serial position, items tended to do so most of the time toward the position of a semantically related item or within the same temporal group. Critically, this impact of semantic knowledge on the pattern of migration errors was not observed anymore in Experiment 2, in which we broke-up the semantic groups, by presenting the semantically related items an interleaved fashion ("leaf"--cloud--"tree"--sky--"branch"--rain). Both semantic and temporal grouping factors may reflect a general mechanism through which information is represented hierarchically. Alternatively, both factors could result from the syntactic and/or semantic regularities that naturally structures linguistic information. These results support models considering direct interactions between serial order and linguistic components of WM.