Difficulties in mathematics learning are a multi-faceted issue and the role of language in it is crucial. Language of mathematics includes mathematics text, discourse in classroom and language used in evaluation situations with its specific verbal, symbolic expressions, structure and function. Ambiguity arising from the difference between formal language of mathematics and the multiplicity of meanings in everyday language aggravates concerns related to the linguistic factors in mathematics learning. A distinction between difficulty from lack of knowledge of concepts and processes of mathematics and difficulty arising from fluency of expressing such skills in appropriate language is often vague. Hence, difficulties faced by students in relation to language of mathematics in different vernacular contexts is to be thoroughly studied for helping teachers to successfully impart the process of mathematization. This paper reports a preliminary diagnostic investigation through descriptive survey, with a mathematical language test to identify difficulty arising from language related aspects of mathematics learning in Malayalam language at elementary level. The sample consisted of 200 students in 7th standard (age 11-12) of Kozhikode and Malappuram districts of Kerala. Association between perceived difficulty in learning Mathematics and difficulties in mathematics language is also analysed. Excluding semantics, all others components of language of Mathematics are highly difficult. Furthermore, difficulties generated by all the components of Mathematical vocabulary, Morphology and pragmatics are contributing to perception of maths as a difficult subject. Unquestionably, the challenges in learning mathematics go beyond the language issues, but the linguistic challenges need to be addressed for students to be successful in attaining aims of mathematics learning.