AN EFFECTIVE ANALYSIS AND EVALUATIONS ARE KEY TO USE EXISTING AND DEVELOPING NEW KNOWLEDGE

Abstract:   The main aim of the paper is to explain the best approaches to develop and enhance analysis and evaluation in academic programmes and a real business world. The research is based on direct and indirect observation and experience in the last forty years working in the Former Yugoslavia and the United Kingdom. The main questions are how and what are the best methods to develop the student’s higher-level skills, including analysis and evaluation, and secondly, how to use these skills in academic and professional context.

The paper recognises that analysis and evaluation are part and prerequisite to other higher-level skills which include conceptualisation and critical thinking, justification, synthesis and creativity. Research conclusions are founded on qualitative analysis. The main value of the paper is to shows relevant and valid academic practice and practical implementations to cultivate and boost higher level skills.

The content will define the key terms, practical steps to impose use of analysis and evaluation, recommended by the Quality Assurance Agency – QAA (2018), supported by strong endorsement of Credit Level Descriptors for Higher Education set up by South East England Council – SEEC (2016), completion of an ethical form prior to any research proposal, project or final dissertation, and finally, an effective check and control of plagiarism using, for example,  Turnitin (www.turnitinuk.com). In parallel to this, HE institutions must rigorously implement these standards within academic documents such as Module Descriptor Definitive Document (2018); Assignment (content, structure and required elements) and Exam questions; Content and guidance within assessment criteria; Mark descriptors; and Assessment and feedback to students.

The paper will show how the above-mentioned documents support and execute use and development of analysis and evaluation using a real example and evidence. Apart from the academic relevance of analysis and evaluation some practical guidance will be given how to use them in holistic and range of different tools and techniques in everyday life, problem-solving and decision-making process. Some remarks to other higher-level skills such as ethical awareness and application, interpersonal and communication skills, synthesis and original research will be given in strict context to explain relevance and importance of analysis and evaluation.

Keywords: Higher level skills; Analysis and evaluation; Critical thinking; Cognitive skills, Credit level descriptors; Undergraduate grading criteria; The 21 centuries skills; Analysis methods and techniques.

The fields: Social science and Humanities.

Introduction 

The paper will start with the definition of the keywords, including higher level skills, analysis and evaluation and how these skills are supported and defined as a unique range of attributes by Quality Assurance Agency – QAA and South East England Council – SEEC.  The next part focuses on good academic practice and documents (module guide, assignment, mark descriptors and assessments) which secure the effective and efficient use of analysis and evaluation. Finally, some practical impact of an analysis and evaluation of employability skills are shown.

 

1. Higher education institutional support of higher level skills and its implementations 

It is important at the beginning to define the main terms and keywords. Higher level skills are initially introduced by Benjamin Bloom back in 1956 which defines and categorises lower order and higher order thinking skills. See more: Anderson and Krathwohl (2000, and 2016):  TES Connect Digital Publishing (2013); Gershon (2015).

The system contains six levels, which are arranged in hierarchical form, moving from the lowest level of cognition (thinking) to the highest level of cognition (or from the least complex to the most complex):

• Knowledge Thinking skills (Lower order)
• Comprehension
• Application
• Analysis
• Synthesis (Creation)
• Evaluation Thinking skills (Higher order)

Thus, it is very important, especially for employability, to develop higher order skills, such as the ability to analyse, judge and implement knowledge in real business and life situations.

Analysis can be explained in many ways depend on context, academic and professional complexity and field of research. Analysis as a detailed examination of anything complex, such as an organisation, problem, society or business environment to understand its nature or to establish its essential content, structure, causes or influences to change, do nothing or improve.

Evaluation is the ability to establish importance or relevance of something. It is a judgement, check and critique about the subject content or factors, such as amount, size, influences or value and assessment of their relevance and validity to an area of knowledge, applications or to an area of research.

The above skills are clearly defined by QAA (2015) as a unique range of attributes for all academic levels and the wide range of different educational and professional areas.

SEEC (2016) describes credit level descriptors as ‘the level of knowledge, complexity, and autonomy expected of a learner on completion of a defined and bounded learning activity such as a module or programme of learning.’ The descriptors are grouped under five headings, including setting, knowledge and understanding; cognitive skills, performance and practice, personal and enabling skills. The paper will describe and assess only an analysis and evaluation and some aspects of synthesis and creativity. The next table will sum -up and shows the importance of analysis and evaluation across all educational levels.

 

Table 1 SEEC Descriptors and comparative summary for analysis and evaluation

Level 3	Level 4	Level 5	Level 6	Level 7	Level 8
Analyses a range of information using pre-defined principles, frameworks or criteria.	Judges the reliability of data and information using pre-defined techniques and/or criteria.	Analyses a range of information, comparing alternative methods and techniques.  Selects appropriate techniques/criteria for evaluation and discriminates between the relative relevance and significance of data/evidence collected.	Analyses new, novel and/or abstract data using an appropriate range of established subject-specific techniques.  Judges the reliability, validity and significance of evidence to support conclusions and/or recommendations. Suggests reasons for contradictory data/results.	Undertakes analysis of complex, incomplete or contradictory evidence/data and judges the appropriateness of the enquiry methodologies used. Recognises and argues for alternative approaches.	Undertakes independent analysis or evaluation, managing complexity, incompleteness of data or contradictions in areas of knowledge.

Source: Credit Level Descriptors for Higher Education – 2016, available at http://www.seec.org.uk/wp-content/uploads/2016/07/SEEC-descriptors-2016.pdf [Accessed: 06 August 2018]

The above table shows that analysis and evaluations are part of every level and must be persistently, adequately and rigorously used in a programme and each module in a range of different teaching and learning practices.

 

2. Good academic practice and documents are sources to provide effective use of analysis and evaluation

There are a lot of academic documents which implement above academic descriptors, principles and recommendations. Module Descriptor Definitive Document – MDDD is the first document which content must be rigorously followed. The main MDDD sections are:  1. Module Code; 2 Module Title; 3. Credits; 4.  Validation period; 5. Status (Validated); 6. Subject Board; 7. Academic Level; 8. Study Period (Semester); 8. Location: 9. Prerequisites and co-requisites; 10. Associated programmes; 11. Content (indicative); 12. Module aims; 13. Intended Learning Outcomes (13a. Knowledge and Understanding and 13b. Skills and attributes) 14. Delivery mode (Classroom Based; Distance; Flexible; Study Abroad; Work Based Learning); 15. Teaching and Learning Activities (Teaching & Learning activities will comprise: Case Studies, Critical Reflection, Directed Study and Reading, Independent Learning, Lectures, Peer Group Study, Presentations, Seminar Groups, Tutorials, Virtual Learning Environment, Workshops); 16. Assessment (formative & summative); 15. Indicative of learning (Books; Websites; Journals and Magazines). See more, Marjon (2018).

The document provides lecturers and students with an integrated and holistic introduction to module or subject. This document provides the foundation for design, standardisation and unification across all modules within a programme. The modules are defined including structure, content, delivery and assessment. This document must be strictly applied and cannot be changed without a quality standard approval.

The above content must be implemented in a written module guide in which every part must be specifically addressed to specific details and origins of the subject.

A module aims should be defined to address level 6, 7 or 8 listed in table 1. For example, the aims of strategic management module are to enable students to: (1) Develop an understanding of strategic management theories, concepts and applications; (2) Understand the main elements of corporate strategy including the strategic position, strategic choices and strategy implementations and (3) Analyse and assess effectiveness of different strategic tools and techniques currently used in real business world. Further, the module learning outcomes should address knowledge and understanding using descriptors in table 1. The successful students will typically be able to: (1) Evaluate the current theories, concepts and principles of strategic management, and (2) Understand the influences of marketing, human resources management and finance on strategic management. The skills and attributes are also listed to include higher level skills, such as (1) Synthesise strategic management concepts to management problems in the construction sector, and (2) Demonstrate the ability to express their skills and knowledge in a professional context.

Next and very important document is an assignment which must include a relevant part of the module.  An assignment reinstates specific module learning outcomes with specific, focused and clear assignment brief, submission requirements (type – report or essay; a number of words +/- 10 percent and in which format). The most important part of the assignment is defined as criteria for assessment and allocation of the points. For example, marks awarded for strategic management assignment will be:  Clarity, structure and methodology (10%); Business environment assessment – PESTEL, 5 Force model (30%); SWOT & TOWS including analysis and evaluation (30%); Stakeholder analysis, mapping and strategic objectives (20%); Language and referencing (10%).

Defined criteria and allocation of points are very important guidance and road map to help students to complete the tasks. The criteria recall again the importance of higher level skills (analysis, evaluation and synthesis) which are awarded by a huge percentage.

This is professional, well-organized and resourceful marking which secure clarity, fairness and ethical perspective in both academic and professional field. It is an effective method to eliminate the prevailing approach in some HE institutions were marking is generic and subjective assessment without specific and focus feedback.

An important part of the assignment is standardised an undergraduate and postgraduate grading criterion for all different forms of exams including reports, essays, posters, presentations, team and group works. The main marks with 100 points awarded are:  Little or nothing of merit (1 – 19); Clear Fail (20 – 29); Marginal Fail (30-39); Satisfactory (40-49); Good (50-59); Very good (60 – 69); Excellent (70 – 79) and Outstanding (80 – 100) percent.

An excellent mark must include subject insight & application combined with breadth, depth & integration of literature and research data into work. An outstanding mark must include elements of originality in both breadth, depth of knowledge, integration of literature and original research based on primary and secondary research. At the same time, both marks must possess strong elements of discussion, analysis (quantitative and qualitative), critical evaluation and or reflection. The sum of an allocation of points and marks descriptors are given in the next table.

Table 2 Undergraduate grading criteria for top outstanding and excellent marks for reports

REPORT	Presentation & structure	Use & presentation of Harvard Referencing	Content/ Terms/ Findings/ Definitions/ Calculations	Business Application & Integration of Data/Literature	Discussion /Analysis /Critical evaluation &/or Reflection
Task details	Follows report structure & keeps to word limit of …	Follows Harvard style for in-text citation & Reference List, and Use a minimum of …  sources	…Content included – specify task requirements as in module guide & coursework guidance	Integration & application of information – from coursework guidance /module guide	Line of argument, development of discussion and instructional verbs to suit the task & level
100 points	/10	/10	/20	/30	/30
Outstanding80-100	Outstanding…  Presentation & report structure, with numbered paragraphs, list of contents/figures &appendices.Articulate & fluent academic writing style with ideas cross-referenced. No grammatical / spelling errors.	Outstanding… Standard of referencing within text & consistent use of the Harvard referencing system.The accuracy of in-text references & full details shown in the Reference list.	Outstanding… Exploration of the topic showing excellent knowledge & understanding through thorough & appropriate research.Impressive choice and range of appropriate content.	Outstanding… Business insight & application.Breadth, depth & integration of literature/data into work.	Outstanding…  Level of discussion/analysis/ critical evaluation &/or reflection.Highly developed/ focused work.
Excellent70-79	Excellent … Presentation & report structure, with numbered paragraphs, list of contents/figures, appendices & cross-referencing.Articulate & fluent academic writing style. Only a minor error.	Excellent… Standard of referencing within text & consistent use of the Harvard referencing system.The accuracy of in-text references & full details shown in the Reference list.	Excellent … Level of knowledge & understanding demonstrated. Evidence of appropriate reading. Covers all relevant points & issues.	Excellent … Business insight & application.Breadth, depth & integration of literature/data into work.	Excellent…  Level of discussion/analysis/ critical evaluation &/or reflection clearly developing points in the appropriate way with thorough consideration of all possibilities.

Source: Grading criteria for Hertfordshire Business School (Reports). Hertfordshire University. Hatfield. The United Kingdom.  

The above table shows clearly grading criteria, its descriptors and allocation of the points. Kings College London (2018) emphasises that “… marking criteria provides guidance on the overall standards expected at different grade bands but discipline-specific criteria may be needed in order to ensure that marking decisions are consistent, fair and transparent to both staff and students.” The absence of clearly well-defined marking criteria, lack of clarity and rise of the marked assignments and exams with highest mark ten (10) are the most common problem.  In many occasions assessing the postgraduate papers, it was difficult to find any attribute of a sophisticated understanding of the topic, with a high degree of competence, excellent usage of relevant literature, theory and methodology combined with evidence of originality.  Thus, this practice must be changed.

Kings College London (2018) describes distinction criteria (90 – 100) in three categories: (1) Understanding – Advanced, in-depth, authoritative, full understanding of key issues with evidence of originality; (2) Depth of knowledge – Complex work and key issues analysed a wide range of sources used selectively to support argument and discussion, and (3) structure – Coherent and compelling work logically presented. General description for highest mark (90-100) is insightful work displaying in-depth knowledge. For research dissertation and project: publishable quality, outstanding research potential, originality and or independent thought, ability to make informed judgments and highest standards of presentation.

3. The practical impact of an analysis and evaluation to employability skills 

World Economic Forum and Boston Consultancy Group (2016) publishes New Vision for Education: Fostering Social and Emotional Learning through Technology with a strong message – to thrive in the 21 centuries, students need more than traditional academic learning. The students require 16 skills assembled in three different ranges: (I) Foundational literacies, which shows how students apply core skills to everyday tasks, includes: 1. Literacy; 2. Numeracy; 3. Scientific Literacy; 4. ICT Literacy; 5. Financial Literacy and 6. Cultural and Civic Literacy; (II) Competencies which shows students approach complex challenges, includes: 7. Critical thinking and problem-solving; 8.  Creativity; 9. Communication; 10. Collaboration; and (III) Character Qualities, which shows how students approach their changing environment, includes: 11. Curiosity; 12. Initiative; 13. Persistence and grit; 14. Adaptability; 15. Leadership and 16. Social and cultural awareness.

Therefore, World Economic Forum (2018) lists the top ten skills that will be most desired by employers by 2020:

1. Complex problem-solving
2. Critical thinking
3. Creativity
4. People management
5. Coordinating with others
6. Emotional intelligence
7. Judgment and decision-making
8. Service orientation
9. Negotiation skills
10. Cognitive flexibility.

Without a great deal of elaboration, the above-listed skills clearly demonstrate the essence and impact of analysis and evaluation to enhance listed students’ skills.  It is the evident impact on complex problem-solving, critical thinking, judgment and decision-making and cognitive flexibility. The report shows that 36% of all jobs across all industries will require complex problem-solving abilities as a core skill by 2020.

Any practical problem, implement of change, decision making, to address the impact of the main drivers to reconstruct, develop or improve must start with valid and relevant analysis supported by evidence and primary and original research. Thus, analysis and evaluation become vital for academic and professional applications. The main requirements for any assignment or assessment within professional education provided by Institute for Leadership and Management, the Chartered Institute of Marketing or Chartered Management institute must start with effective and efficient analysis supported by evidence and fact. The next stage is an evaluation, recommendations and justification.

It is understandable that the main players in public sector and industry such as CIMA (2007), DFID (2003), USAID (2014), WB (2007), Johnson &Whittington (2017) provides detail guidance how to use different tools and techniques to complete holistic, fact-based and logical analysis. Without any question, it is the ultimate objective to teach the students on all academic and professional levels to analyse and evaluate more.

Conclusion 

The HE institutions must be open, ready to cooperate and implement the best benchmarking practice across in the world to develop higher-level skills, especially employability skills for 21 centuries. The paper finds that is very important to develop, nurture and demonstrate effective use of evidence-based analysis and evaluation across all modules, research and learning activities. Development of higher-level skills must be consistent and holistic.

The purpose is to develop the student’s ability to critical thinking, judgment and abilities to apply and use gained knowledge. Instead, of memorised, descriptive and generic knowledge without any based evidence, fact or practical research. Persistent use of analysis and evaluation, especially on higher levels must be seen as a process to improve employability skills and independent abilities of the students to assess, analyse, judge and implement knowledge in real business and life situations.

 

Reference list:

 

1. The UK’s Quality Body for Higher Education (2018), available at http://www.qaa.ac.uk/ [Accessed: 04 August 2018]
2. Credit Level Descriptors for Higher Education – 2016, available at http://www.seec.org.uk/wp-content/uploads/2016/07/SEEC-descriptors-2016.pdf [Accessed: 04 August 2018]
3. MODULE DESCRIPTOR Definitive Document (2018), available at https://www.marjon.ac.uk/about-marjon/institutional-documents/module-descriptors/bma-modules/BMAH02-Entrepreneurship-&-Small-Business-Man..pdf [Accessed: 04 August 2018]
4. Bloom’s taxonomy, available at https://tlc.lincoln.ac.nz/blooms-taxonomy/ [Accessed: 08 August 2018]
5. TES (2013) How to use Bloom’s Taxonomy in the classroom, available at https://academicamentoring.com/wp-content/uploads/2017/01/How-to-use-Blooms-Taxonomy-in-the-classroom.pdf [Accessed: 08 August 2018]
6. , M. (2015), How to use Bloom’s Taxonomy in the Classroom: The Complete Guide: Volume 8 (The ‘How To.’ Great Classroom Teaching Series). Kindle edition.
7. QAA (2015) Characteristics Statement for Master’s Degree, available at http://www.qaa.ac.uk/docs/qaa/quality-code/master’s-degree-characteristics-statement.pdf?sfvrsn=6ca2f981_10 [Accessed: 08 August 2018]
8. QAA (2015) Doctoral Degree Characteristics, available at http://www.qaa.ac.uk/quality-code/the-existing-uk-quality-code/part-a-setting-and-maintaining-academic-standards [Accessed: 08 August 2018]
9. Marjon (2018), MDDD, available at https://www.marjon.ac.uk/about-marjon/institutional-documents/module-descriptors/gmb-modules/GMBM02.pdf [Accessed: 10 August 2018]
10. Kings College London (2018) Taught Postgraduate Generic Marking Criteria, available at https://www.kcl.ac.uk/governancezone/Assets/Assessment/Marking%20Criteria,%20Taught%20Postgraduate.pdf [13 August 2018]
11. World Economic Forum and Boston Consultancy Group (2016) New Vision for Education: Fostering Social and Emotional Learning through Technology, available at http://www3.weforum.org/docs/WEF_New_Vision_for_Education.pdf [13 August 2018]
12. World Economic Forum (2018) What are the 21st-century skills every student needs?, available at https://www.weforum.org/agenda/2016/03/21st-century-skills-future-jobs-students/ [13 August 2018]
13. CIMA – Strategic Analysis Tools (2007) available at https://www.cimaglobal.com/Documents/ImportedDocuments/cid_tg_strategic_analysis_tools_nov07.pdf.pdf [13 August 2018]
14. DFID – Tools for Developemnt (2003) available at http://webarchive.nationalarchives.gov.uk/+/http:/www.dfid.gov.uk/documents/publications/toolsfordevelopment.pdf [13 August 2018]
15. USAID – Types of Business Analysis (2014) available at https://www.usaid.gov/sites/default/files/documents/1868/597sah.pdf [13 August 2018]
16. World bank (2007) Tools for Institutional, Political, and Social Analysis of Policy Reform, available at http://documents.worldbank.org/curated/en/434581468314073589/pdf/390220Tools0fo101OFFICIAL0USE0ONLY1.pdf [13 August 2018]
17. G., Whittington. R., (2017) Exploring Strategy: Text and Cases. Pearson education.London.