Some readings (& podcast) putting Randomized Control Trials (RCT)s into perspective

Like incentivised laboratory experiments Randomized Control Trials (RCTs) are all the rage in economics.  RCTs are commonplace in the health sector starting with Pasteur’s first controlled trials 200 years ago. While application of RCTs to social sciences is relatively recent.

However, by their very nature social sciences involve researching social groups and networks where information is distributed and co-ordinated with relative ease and frequency.

This creates a unique problem for RCTs in social research because it is very difficult to construct experiments that are able to completely seal information within evaluated units.  Importantly, the closer the social networks of individuals the more likely there will be information contamination and that individuals in the ‘control’ condition will act on this information. Continue reading

Problem of rational adaptive behaviour in at-risk youth

Heller, S. B., Shah, A. K., Guryan, J., Ludwig, J., Mullainathan, S., & Pollack, H. A. (2015). Thinking, Fast and Slow? Some Field Experiments to Reduce Crime and Dropout in Chicago (No. w21178). National Bureau of Economic Research.

ABSTRACT This paper describes how automatic behavior can drive disparities in youth outcomes like delinquency and dropout. We suggest that people often respond to situations without conscious deliberation. While generally adaptive, these automatic responses are sometimes deployed in situations where they are ill-suited. Although this is equally true for all youths, disadvantaged youths face greater situational variability. This increases the likelihood that automaticity will lead to negative outcomes. This hypothesis suggests that interventions that reduce automaticity can lead to positive outcomes for disadvantaged youths. We test this hypothesis by presenting the results of three large-scale randomized controlled trials (RCTs) of interventions carried out on the south and west sides of Chicago that seek to improve the outcomes of low-income youth by teaching them to be less automatic. Two of our RCTs test a program called Becoming a Man (BAM) developed by Chicago-area non-profit Youth Guidance; the first, carried out in 2009-10, shows participation improved schooling outcomes and reduced violent-crime arrests by 44%, while the second RCT in 2013-14 showed participation reduced overall arrests by 31%. The third RCT was carried out in the Cook County Juvenile Temporary Detention Center (JTDC) in 2009- 11 and shows reductions in return rates of 21%. We also present results from various survey measures suggesting the results do not appear to be due to changes in mechanisms like emotional intelligence or self-control. On the other hand results from some decision-making exercises we carried out seem to support reduced automaticity as a key mechanism. Continue reading

Survey demographics – School Choice

The surveys generated a representative spread of parent backgrounds, including age, education level and household income.

parent demographics2

The survey also generated a good spread of secondary school types attended by children. A situation where children in one family attend more than one school type (Mixed) is likely to occur because of 1) individual children gaining selective entry or Continue reading

‘Determinants of Parent School Choice’ Online Survey – the Results

This post will provide updated links to results as I post them. Posts will initially focus on straight forward results associated with specific questions, before proceeding onto more complex statistically analysis of relationships between questions. Continue reading

‘Determinants of Parent School Choice’ Online Survey

I’m a PhD student at RMIT investigating the underlying motivations of parent school choice from an economics perspective.  The objective of this research is to understand the behavioural decision rules used by parents in choosing schools for their children.  This survey is anonymous and may take up to 30 mins to complete. A brief bio about myself can be found here.

——————————————– The survey is now closed —————————–

The key focus of this survey is the idea that education is an investment in a child’s future. Consequently, investments in a child’s education (such as school choice) are generally considered to be governed by the same general economic principles that we see in similarly complex decision making. However, parents usually make these decisions with limited time and resources.  This survey seeks to test this assumption by understanding the relationship between school choices and economic behaviour linked to risk and social preferences.  We draw on insights from behavioural economics to test whether decision behaviour is consistent across different types of choices and different contexts in which choices are made. This survey follows on from my qualitative research into school choice (Victoria, Australia).  It also draws on some interesting observations coming out of the linguistic analysis of these qualitative interviews which indicated the potential existence of distinct economic decision types influenced by economic risk and social preferences.  The survey also draws inspiration from Jonathon’s Haidt’s research on how ‘Liberals and conservatives rely on different sets of moral foundations’. The other investigators for this research project are my PhD supervisors Professor Jason Potts, Dr Foula Kopanidis from RMIT’s School of Economics, Finance & Marketing and the research has been approved by RMIT’s Human Research Ethics committee (No.18945).

Experimental evidence of ‘Intergenerational Egalitarianism’ – Hauser et al 2014: Cooperating with the future

Hauser, O. P., Rand, D. G., Peysakhovich, A., & Nowak, M. A. (2014). Cooperating with the future. Nature, 511(7508), 220-223.

ABSTRACT: Overexploitation of renewable resources today has a high cost on the welfare of future generations. Unlike in other public goods games, however, future generations cannot reciprocate actions made today. What mechanisms can maintain cooperation with the future? To answer this question, we devise a new experimental paradigm, the ‘Intergenerational Goods Game’. A line-up of successive groups (generations) can each either extract a resource to exhaustion or leave something for the next group. Exhausting the resource maximizes the payoff for the present generation, but leaves all future generations empty-handed. Here we show that the resource is almost always destroyed if extraction decisions are made individually. This failure to cooperate with the future is driven primarily by a minority of individuals who extract far more than what is sustainable. In contrast, when extractions are democratically decided by vote, the resource is consistently sustained. Voting is effective for two reasons. First, it allows a majority of cooperators to restrain defectors. Second, it reassures conditional cooperators that their efforts are not futile. Voting, however, only promotes sustainability if it is binding for all involved. Our results have implications for policy interventions designed to sustain intergenerational public goods.

The key insight of this experiment is that a system based on simple democratic rules can overcome the tendency of small groups of people to rationally over exploit resources in the current generation leading to resource collapse.  Given that there will always be some probability that there will be individuals who rationally have no regard for future generations, resource collapse is (almost) certain to occur.  However, the authors show that simple democratic voting rules binding all participants are effective in restraining this rational, generationally selfish, behaviour.  Consequently, resources are sustained over multiple generations of participants.

This paper dove-tails with a two of key areas related to intergenerational investment & the role of government.

Continue reading

Results from school choice interviews – Switching schools

Switching school type – primary to secondary school

Of the 22 parents interviewed, nine families (41%) changed the type of school their children attended between primary and secondary school.  Most of these changes, five families, were from public schools to independent schools.  Two families enrolled their children in selective public schools and two families made the decision to change from a Catholic primary school to a public secondary schools. The change from Catholic public primary schools to public secondary schools was largely motivated by strong preferences for a co-education school environment.  Catholic secondary tend to be predominantly single sex schools.

Switching School - ChildSwitching schools – parents’ and children’s secondary school attended

In general choices followed the academic background of the parents.  Choices where changed either when parent schooling experience was negative or a salience characteristic of the parent school experience was missing.  For example, negative experiences from attending a regional public school leading to a preference independent school education or the absence of co-educational choice at Catholic schools leading to a change to public or independent schools.

Continue reading

A reflection on expected experimental outcomes – Latent Semantic Analysis of interviews

When I first thought of the possibility of using Latent Semantic Analysis (LSA) to analyse my parent interviews exploring school choice my expectation was that the odds of success were slim.  I was familiar with the use of LSA in helping decipher the contextual meaning of words in ancient texts and indicate the likelihood a text was written by a particular person based on their known works.  LSA is one of the methods used to try and identify (speculate) who the ‘real’ Shakespeare was by comparing the works of Shakespeare with the writings of other contemporaries.  But could LSA be applied to interviews investigating economic decision making behaviour and provide meaningful insights? Would it be possible to identify key concepts influencing the decision making process based the common use of key words? Could LSA identify ‘latent authors’ representing distinct, heterogeneous, types of decision making within a society? Noting that the conventional economic wisdom is that society is comprised of a homogenous set of individuals (one type) applying the same decision processes subject to variability environmental conditions (such as wealth) and uncertainty.  To draw on quantum physics – there are no ‘flavours’ or handedness in standard economic theory.

Continue reading

Demographics of parents interviewed about choice of secondary school

The table below provides some detail as to the level diversity of the interviewed parents.  It is important to note that the focus of the study was to explore and understand the decision architecture of how parents choose a school for their children.  Consequently, a degree of priority was given to finding parents who had switched schools from public to private (independent/Catholic) or from private to public in the transition from primary to secondary schooling.

demographics

Non-Catholic private schools in Australia are called ‘independent schools’ because these schools (particularly the elite schools) govern, manage, resource and finance themselves independently of any broader religious or philanthropic affiliation.

In contrast, Catholic schools are managed, financed and resourced centrally in a manner broadly similar to government run public schools but with higher levels of community involvement and direct fee payment associated with the choice of school. While there has been a trend for Independent secondary schools to move from single sex to co-educational, Catholic secondary schools in the metropolitan cities by and large are single sex schools.

Continue reading

Latent Semantic Analysis

An everyday application of Latent Semantic Analysis (LSA) is the Google search engine where words that are semantically/contextually similar are also returned in the search query.  Type in “run” and the search will also pick up “ran”, “runs” and “running”. LSA allows natural language processing of vast collections of data, such as web pages, to provide information about how similar words are related to each other in (semantic) context by converting words into vectors (vectorial semantics) and applying singular value decomposition to the matrix.  In this way, the data itself is used to create a ‘latent semantic dictionary/thesaurus’ which reflects the context of the documents being analysed.

LSA captures the contextual relationships between text documents and word meanings.  Taking into account the context in which words are used is important for linguistic analysis.  The contextual meaning of words change over time and across social groups.  An example of the importance of context is how the meaning of ‘terrific’ changes over time.  Latent semantic analysis of documents from the second half of the 19th century would show ‘terrific’ as similar to ‘horror’. While documents from the second half of the 20th century would show ‘horror’ as now being the opposite of ‘terrific’.

Continue reading

Isaac Asimov quote on scientific discoveries

“The most exciting phrase to hear in science, the one that heralds the new discoveries, is not ‘Eureka’ but ‘That’s funny…'” – Isaac Asimov

like, why is fundamental discounting behaviour (whether Rational or Behavioural) common to most multi-period investment & consumption decisions missing from decisions parents make about investments in their children’s education?  The ‘wait a sec, where is it?’ moment.

A discrete choice experiment : school choice

As part of my PhD research project “Modeling the hardest decision parents will make: School Choice” I will conducting a Discrete Choice Experiment (DCE) to understand how Australian parents apply behavioural decision rules in choosing schools for their children, and how choice sets are evaluated to find the ‘best choice’.

This research will help understand why Australian parents are spending an extra $5.4billion more per year in private school fees when socio-economic sorting across public schools would lead to the same academic outcomes.

We also hope to gain insight in to why some recently opened private schools have failed, leading to large losses, despite the strong willingness of parents’ to pay for private schooling.

This project will also look at the influence of a parent’s socio-economic background on the type and strength of preferences. Our results will have important social equity implications for understanding how wealth, occupation and prior education affect a parent’s choice of school. This will be the 1st time a discrete choice experiment has been applied to school choice even though it is an experimental approach applied extensively in other quasi-market public good areas of health, the environment & transport infrastructure. Previously DCE hasn’t been applied to education due to the inherently endogenous behaviour of educational choice attributes, which this project resolves.

If parents are not making classical rational choice decisions, this research will have important implications for economics & education policy that have not previously been identified.

Determinants of parental school choice: A Qualitative Study

I’m currently undertaking a qualitative study into the determinants of parental school choice.  The study comprises 22 semi-structured exploratory interviews of Australian parents,  principally from Melbourne with some from regional Victoria.  Parents come from diverse backgrounds of educational history, educational choice of school type, and cultural.

Following from George Shackle that for choice there need to be alternatives, the socio-economic backgrounds of the parents interviewed are broadly middle socio-economic, from low to high middle class.  For the very wealthy there is no alternative to the ‘best’ and for the low socio-economic parents income & behavioural constraints mean that there are no alternatives to their default choice.

Australia is a particularly interesting country to investigate the decision architecture of how parents choose a school for their children due to the absence of strong racial (USA) or social-class preferences (UK).  Race and social-class preferences are present in Australia but are not strong enough to completely outweigh other preferences that it is difficult to differentiate preferences associate with teacher quality, student personal development, discipline and community.  Race in the USA and social-class in the UK have become dominant proxies for these more differentiated preferences leading to simple binary choice decision making.

Having a range of differentiated preference attributes allow a deeper investigation into how social & family human capital investment preferences are traded-off between each other and the decision strategies that are being used.  When there a number of competing preferences which are quite varied, it can be very interesting to find some behaviours that should be present but is consistently missing from the interviews.  When choices are binary by proxy it is difficult to find decision structure and gaps.

Parental choice is extraordinarily complex being both inter-generational and inter-temporal in nature. It is subject to parental income, time and regulatory constraints. Choice is subject to high levels of uncertainty over very long time frames. Choices are path dependent, in most case irreversible and subject to imperfect information. Individual choice is also very context dependent, subject to the experiences of parents, their expectations of the future, a duty to their children and emotional attachment.

Where there is a common curriculum, choice is not about the quality of ‘education as knowledge’ per se. At a very high level, education choice is about quality of instruction, teacher quality or program choice, or the quality of the learning environment effected by student peers and culture. Individual preferences of parents are reflected in the type of school they choose for their child to attend. Choice may be a default choice based on constraints, or a choice exercise within a type of schools such as different public schools, or across school types.

The main education choice that this study focuses on is choice of secondary school by parents for their children.  Choices in Australia are: independent schools (generally Protestant religion aligned), Catholic schools, government public schools (entry determined by residential boundary), and government selective public schools (academic, music, sport etc.).  The state of Victoria also has an accelerated learning program in some publics schools which are not constrained by location of residence by require passing an entrance exam.

A comprehensive review of the Australian school sector can be found in the Gonski Review.

It is important to note that unlike the USA, funding of schools in Australia is from broad based taxes (universal vouchers approach) and not aligned with local property taxes.

%d bloggers like this: