Analyses | Education and human resources | 2016-03-07

TSA: Test Students Accurately



After months of consultation, the Education Bureau (EDB) of the HKSAR published a report[1] written by the Coordinating Committee on Basic Competency Assessment and Assessment Literacy regarding upcoming enhancements to the oft-criticised Territory-wide Assessment (TSA).[2] The report made several recommendations to the EDB regarding the implementation of the TSA, which were largely accepted by the Bureau.As a preliminary measure, 90% of primary school students will not be required to write the TSA for Primary 3 this year. Instead, 50 primary schools will be invited by the Hong Kong Examinations and Assessment Authority (HKEAA) to write a try-out study for the TSA which will feature fewer and more straightforward questions. Invited schools will also be given a choice to opt out of the test.

Teachers and parents who have called for the TSA to be abolished will likely remain dissatisfied with the report’s recommendations.[3] Although the report acknowledged the seriousness of the concerns raised by the community regarding the TSA, the EDB reaffirmed that many of these concerns are based on misunderstandings of the test. Thus, while the TSA could certainly be improved, the Bureau still believes that the TSA has value because it provides the Government and schools with useful information on student standards that can identify educational priorities to enhance student learning.

Bureau officials have repeatedly stated that the test cannot assess individual students, nor is the test meant to assess skills outside of the basic curriculum. Therefore, teachers should feel no pressure to drill students or compel students to do TSA-related homework outside of class.[4] In the report, the Committee advised that the EDB address over-drilling by improving their communication with the public regarding the original intent of the test. To lessen the pressure on schools to improve TSA scores, the report also suggested introducing simplified reporting formats for schools to choose from when receiving TSA scores from the Bureau, while enhancing training for teaching staff in areas of curriculum development and providing schools with TSA-related professional support.

One of the report’s long-term recommendations was for the EDB to draw reference to assessment practices in other regions, “particularly their approaches in using assessment data for devising measures to support teaching.” It may be useful to evaluate recent developments with school assessments in the United States to discern if any lessons can be applied to Hong Kong.

Standardised testing: The American comparison

On Dec 10, 2015, President Barack Obama signed the “Every Student Succeeds Act.”[5] This law replaced the “No Child Left Behind Act” (NCLB), passed in 2001. NCLB was criticised for placing too much emphasis on standardised tests. Under NCLB, each state was compelled to develop mandatory assessments for all students in certain grade levels in order to receive federal school funding. Every student had to write tests administered by the state government along with the National Assessment of Educational Progress (NAEP). 40% of American students took three to five assessments a year.[6]

Average NAEP scores improved after NCLB’s passage. Yet the law also punished schools that failed to meet minimum standards of growth in test scores.[7] States were forced to close down or take over schools that failed to meet minimum standards for five consecutive years. Most of these schools were in low-income communities, and their staff felt like they were penalised unfairly for teaching in impoverished areas. States also complained that they had no flexibility in deciding which schools were “underperforming” and what remedies should be enacted. Obama acknowledged that NCLB policies “didn’t always produce results,” while coercing teachers to “teach to the test.”

Under the new law, while annual tests will still be written from third to eighth grade and once in high school, states now possess the ability to reduce or eliminate standardised tests that they deem to be unnecessary or unhelpful. In Obama’s words, the focus is on ensuring that students write tests “that are worth taking.”[8] States will have the discretion to improve underperforming schools in any manner they see fit, rather than being mandated specific punishments by the federal government. They have also been given the flexibility to assess school quality by taking into account factors not limited to test scores or purely academic criteria. Finally, Obama recommended that states limit test-taking time to no more than 2% of the year, so that teachers can spend more time engaging in student learning.[9]

Thus, though it is true that the United States Government continues to assess schools with test scores, efforts have been made to reduce the importance of standardised tests and improve their implementation. President Obama has encouraged states to reduce overall test-taking time while scrapping tests that do not improve teaching. So, while cancelling the TSA outright could be considered a regression as compared to the United States, failing to modify the test in a way that improves its accuracy and reduces the stress it imposes on students may constitute a regression as well.

“Teaching to the Test” and Test Score Validity

In order for the TSA to provide “quality and reliable data,” TSA scores must be valid.[10] In other words the scores must correspond with students’ basic competency levels in English, Chinese and mathematics. If this is not the case, teachers will have difficulty discerning strengths and weaknesses in their methods, while the Bureau may be using inaccurate data in providing targeted support to schools.

Unfortunately, rampant drilling and teaching to the test disrupts the ability for the Bureau to make well-informed assessments measuring the level of mastery students have achieved in relevant subjects. According to Professor W. James Popham from the University of California at Los Angeles, if teachers prepare for assessments by drilling students in class, this “eviscerates the validity of score-based inferences.”[11] In other words, if teachers use items from real tests to instruct their students using rote memorisation, or if they completely avoid teaching material that is not assessed, then TSA results will overestimate students’ true learning abilities. Teachers will then have difficulty identifying strengths and weaknesses in their pedagogical methods.

While the Bureau claims that students do not have to prepare for the TSA, the Hong Kong Professional Teachers’ Union (HKPTU) reported that 80% of teachers felt that the TSA puts “serious” pressure on staff to improve the scores of their students.[12] The HKPTU also alleged that educational officials from the Bureau would often ask primary school principals to improve TSA results when visiting schools, which in turn compels school principals to demand further preparation from students.[13]

The TSA Report

The recommendations made in the TSA Report should have the effect of reducing stress on students and teachers while retaining the test’s capacity to provide valuable data for schools and the Government. If the report’s suggestions are implemented, the number of questions and texts assessed for English, Chinese and mathematics will all be reduced. Practical writing in the Primary 3 Chinese Language section will only be included in one of the sub-papers of reading to avoid giving undue weight to practical writing. In the Primary 3 English section, items expecting answers in the past tense will be scrapped, and assessment items on basic book concepts will be avoided. Finally, for the Primary 3 math section, only one basic competency will be assessed in each item, and items requiring solving linking problems will be minimised. These modifications should give more time to improve competency in the remaining skill areas that are on the test through classroom instruction rather than through drilling.

Another constructive suggestion in the report calls for the inclusion of a questionnaire survey to collect students’ non-academic data, such as “time spent for extra-curricular activities, learning interests, learning habits and other relevant data.” This idea will promote the assessment’s goals of improving student learning by taking into account the fact that non-academic factors are also important in enhancing teaching methods. Through the collection of this additional data, teachers will hopefully gain a better understanding of the many factors affecting learning performance, such as free time and the level of interest in the subject matter, to provide further assistance for student learning.

One possible recommendation which was not discussed in the report was a switch in the marking scheme from numerical grading to a simple pass-fail result. Former Education Commission chief Antony Leung Kam-chung has supported this reform. He argues that it would end drilling while retaining the ability for the Bureau to identify schools whose students do not meet basic competency levels.[14] A clear majority of students in Hong Kong are already meeting the TSA’s minimum standards, with over 80% of Primary 3 students meeting the standards last year.[15] Thus the Bureau may consider using the TSA primarily to identify the minority of students who do not meet these standards, instead of distinguishing high achievers. Adjusting the test to a pass-fail format could address these concerns.

Since the try-out TSA has yet to be written by participating schools, it is still too early to predict whether all of the concerns raised by the community regarding the TSA will be adequately addressed in the months and years to come. As a preliminary measure, the EDB will succeed in reducing TSA-related drilling by a significant amount by exempting 90% of primary schools from participation this year. In the long run, however, simplifying the test could still lead to drilling if the community and the Bureau are not on the same page. Improving the HKEAA’s communications with the public and enhancing teacher training and professional support will be necessary to ensure that the TSA’s original intent of providing reliable and quality data to enhance student learning and teaching methods can be fulfilled.

1 Report on Review of the Territory-wide Assessment, The Coordinating Committee on Basic Competency Assessment and Assessment Literacy, Education Bureau, February 29, 2016, http://www.edb.gov.hk/attachment/en/curriculum-development/tsa/fullreport.pdf .
2 The TSA has been administered annually for all Primary Three, Primary Six, and Secondary Three students attending public schools since 2004. The examination is meant to assess whether students possess minimally acceptable knowledge in English, Chinese and mathematics for their grade level. It does not assess individual students, but it does help to assess schools.
3 Shorter version of TSA to be tried out in 50 primary schools, EJ Insight, February 5, 2016, http://www.ejinsight.com/20160205-shorter-version-tsa-be-tried-out-50-primary-schools/ .
4 Catherine Chan, TSA for learning, not drilling, News.gov.hk, June 25, 2015, http://www.news.gov.hk/en/record/html/2015/06/20150625_192145.lin.shtml .
5 Joy Resmovits, Obama signs Every Student Succeeds Act, marking the end of an era, LA Times, December 10, 2015, http://www.latimes.com/local/education/standardized-testing/la-me-edu-essa-obama-signs-end-no-child-left-behind-20151210-story.html .
6 Jill Barshay, Education myth: American students are over-tested, Hechinger Report, December 7, 2015, http://hechingerreport.org/24830-2/ .
7 Michael Hansen, Mend it or end it? Looking at the research evidence on NCLB’s legacy, Brookings, December 15, 2015, http://www.brookings.edu/blogs/brown-center-chalkboard/posts/2015/12/15-research-evidence-nclb-legacy-essa-hansen .
8 Laurie Ure and Kevin Liptak, Obama administration announces new testing guidelines, CNN, October 24, 2015, http://edition.cnn.com/2015/10/24/politics/barack-obama-teachers-testing/ .
9 Christopher Doering, Obama plan limits standardized testing to no more than 2% of class time, USA Today, October 25, 2015, http://www.usatoday.com/story/news/2015/10/24/obama-schools-test/74536886/ .
10 Catherine Chan, TSA for learning, not drilling, News.gov.hk, June 25, 2015, http://www.news.gov.hk/en/record/html/2015/06/20150625_192145.lin.shtml .
11 W. James Popham, Teaching to the Test? Association for Supervision and Curriculum Development, March 2001, http://www.ascd.org/publications/educational-leadership/mar01/vol58/num06/Teaching-to-the-Test¢.aspx .
12 廢除TSA,還小學正常的教育生態!「全港性系統評估」(TSA)問卷調查結果公布, HKPTU, June 22, 2015, https://www.hkptu.org/13788
13 Eric Cheung, 70% of teachers oppose compulsory TSA exams in primary schools: survey, Hong Kong Free Press, June 23, 2015, https://www.hongkongfp.com/2015/06/23/new-survey-shows-70-teachers-oppose-compulsory-tsa-exams-in-primary-schools/ .
14 Tony Cheung and Gary Cheung, Hong Kong’s TSA exam marks should be simple pass or fail to lower the stakes for pupils, ex-official says, South China Morning Post, November 30, 2015, http://www.scmp.com/news/hong-kong/education-community/article/1885078/hong-kongs-tsa-exam-marks-should-be-simple-pass .
15 TSA Report 2015, HKEAA, December 2015, http://www.bca.hkeaa.edu.hk/web/TSA/en/2015tsaReport/priSubject_report_eng.html .