A math problem solving strategy that's proven to work

As I said before, there's a tough set of requirements that any problem solving strategy that's worth teaching to students as a singular go-to overarching framework must satisfy. I have yet to see a simple problem solving strategy satisfy all of those requirements, but here below is something that's been proven to work [1].

The Formulaic Action Oriented Problem Solving Strategy

It's "formulaic" because the back-end of the strategy is centered around using formulas to derive numeric solutions. But also, this strategy is "formulaic" because it's step-by-step, bringing students from start to finish of solving a problem.

It's "action oriented" because it critically answers for students the question of "what should I do?"

Many math problem solving strategies are problematic: here's why

(To see what a better problem solving strategy would look like, skip to the bottom... )


Many problem solving strategies in math are problematic in themselves.  Here are some typical problems with them:


(1) Some are too high level or abstract, and thus provide little to no traction for students to get into practically solving problems. Students either don't know what exactly they should do, or they misunderstand the intent of a step in the strategy.

e.g. one strategy calls for the student to describe any "barriers or constraints that may be preventing them from achieving their goal".  Would "the teacher hasn't taught us" be a valid barrier?  (The answer could be "yes", by the way.)


(2) It assumes a pretty smart student, rather than assume a barely passing one.  So the strategy works great for students who would've done fine without your help in the first place.

Different Types of Learning Tasks and Content Delivery Styles

The following lists a number of learning tasks students may engage in by teacher instruction, and thus they are instruments of teaching a teacher may employ.  Notes regarding each are provided briefly to highlight a range of possible tools teachers may use to engage students in learning.  These are arranged generally speaking from easiest to most difficult to employ effectively.

Every single one of these types of tasks and styles has its rightful place in a course --- there is no magic bullet solution.  Exclusive use of any single style or type of task is not necessarily a good thing as there are always trade-offs to be made.  This list is not meant to be a complete or exhaustive listing of all possible learning tasks or their categorization or description, but a way of thinking about them to help create an effective mixture of learning tasks for any course of learning.

Lectures
- teacher-oriented learning task
- active instrument
- students may remain largely passive for task completion, and can passively wait it out

Theory and verbal/textual explanations are delivered by an instructor to students (focuses on "telling" over "showing").  The delivery is often best done scripted and in an organized manner, almost like a audio/visual version of a printed book delivery of theory or textual explanations.  This term describes a type of teacher-oriented activity.

New angle on math education: lessons from product design

Or a user interface for learning math

High school is when many students typically first encounter some of the greatest difficulties in learning what forms the basis for learning post-secondary level math. Those are, therefore, some very critical years in a student's math educational career. If it is important to get more students to consider entering a STEM career (and I'd argue it is [1]), then it is important to consider how math can be taught to these students so they'd be more interested in it, and be better at it later on.

The following is a long piece discussing the problems facing designing a better math educational experience, complete with the principles of design that can be used to solve such problems, closing with some guidelines for how to create a better course.  In principle, all of the following can apply equally well to other subjects, like computing science, or English, but math education is tough, so I’m picking on the hardest as an example.

Math course as a product

Just how do students experience of learning math get formed? Rather than looking at it in a traditional teacher's mindset, let's look at it in terms of a commercial product designer's mindset. That means we see students as interacting with a complete, full-fledged math education product that promises students that they'd get what they want.

Complex numbers, exponentials, and the 2D plane: geometric and physical intuitions leading to Euler's Formula

**And why polar coordinates turns out to be really useful, and not just for tracking down polar bears.**

Today's goal is to re-imagine the two-dimensional plane, 
to develop a better understanding of how the 2D plane
relates to complex numbers and the complex exponential functions.

We'll do so using mainly our physical and geometric intuitions,
with some basic high school math as our only support.

So we'll need nothing more than high school algebra: i.e.
rules of exponents (i.e. Sum of Exponents rule),
rules of logs (i.e. Sum of Logs rule),
knowing how to use the exponential and 
logarithmic functions with real numbers,
knowing basic trigonometry around the unit circle,
and maybe a little about how angles are measured in radians.

There's no need to know anything about complex 
numbers or vector math.  In fact, we don't even need to know anything about
quadratic equations and how calculating its roots can give us imaginary numbers.

Best of all, by the end, **we're going to re-discover Euler's Formula and 
Euler's identity using only our physical and geometric intuitions** --- no power 
series, and no calculus.

Let's start!

## Grid View: Coordinates

The usual way to think of the 2D plane so that we can give every point
on the plane an "address" is to draw the x- and y-axis, then draw grid
lines everywhere.  This is the **grid system** that we'll start with.

![Standard cartesian plane grid system with x and y axis](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh3g6uNYM5hpFLcYf9hG9cZ1RNN6q6UQIKGA902iGtaubcEKKTEUEczU1T5Z3kPGdsLX9FgUELGUfFGykdOQUCuUUimSQlKVLE43c6cRFUpV5sRQ1nv7EZ1yuzzsWHiltBTBuLH2ZCOpEw/s1600/1_grid.txt.png)

Then every point can be addressed by a rectangular coordinate
\\((x,y)\\) as usual.  If all we know is standard grade-school math,
then we can't really do much with these coordinates.  That means there's no
arithmetic of coordinates in standard grade-school, i.e. so we can't
add or subtract coordinates from each other.

Rectangular coordinates are perfectly fine for giving each point on the
2D plane an address though.

![Standard cartesian plane grid system gives every point an x and y coordinate component](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhvSPWV2REoaUOoB5zK9EClhwaShc8Anbm8j3FUq3vZeJFn_potwDTCMMtKjPWcQTW_L_9Ri9HJWMLaOpmttaYcNGHU631KcmwHTZZryxuQ-VaSXgkndyqtmBti-NYKC2ExlKM2NMVhiZM/s1600/2_xy_pt.png)


## Rectangular View: Right and Up

We can **create** a kind of arithmetics of coordinates though
if we adopt a geometric view of the 2D plane rectangularly:

- Define the **rt** unit of measure as the ruler pointing 
  rightward of length 1.

- Define the **up** unit of measure as the ruler pointing
  upward of length 1.

![Define rt and up as units of rectangular view measures in the grid](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiqGvFpuFwB8Ap6eO04TWrvhOk2tPAbGRzqbtj23F8HUoNp1CYhZpBEMwhwFQk55G1xvEcbV-iAMmsCf3CHsgzg8-oKQV45xa0TjnijV5pXe-bGMvnZKgTGrwmO6cgrBhrId1bKxxtg4fg/s1600/3_new-units-rt-up.png)

Then every point can be addressed by starting at the
origin \\((0,0)\\), and going a certain multiple of **rt**
plus going a certain multiple of **up**.  I.e.:

$$(x,y) = x \cdot\mathrm{rt} + y \cdot\mathrm{up}$$

![Example of adding 3 rt and 2 up to get coordinate at x equals 3 and y equals 2](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjJpf-so6MDiBcEMbb5sTp_elqlDucZ7TEqgNRWcNALvqMYJRMb_hFzkCxleioEj1jUWQX81nnu0DPmxU-257Ix5_TSemi7_l0q0HweAVFjO7QYWmVTtGsZLxkJcfxkCqco6iLGTEJPCNQ/s1600/4_example_rt_up_addition.png)

That is an abuse of the "+" symbol to mean "go one direction,
plus go another direction", but that's what we're trying to
do: make arithmetic sense out of things that doesn't 
naturally have arithmetic meaning.

Notice though that the same point in the same spot of the 2D plane
now has two ways of being addressed!  The grid view gave us **coordinates**, vs.
the rectangular view that gave us an arithmetic **sum** of the **rt** and **up**
units.  We now know the two different addresses are "equal" only because they
are addressing one and the same point in the 2D plane!

We haven't, however, defined a multiplication symbol for mixing
the two units **rt** and **up** together yet,
because what could it possibly mean to multiple a **right**
with an **up** direction?

Computer Science: a diluted term, soon to be meaningless

I just saw a grade 6 school teacher's course web page.  He teaches a "Computer Science" course wherein students learned to type and format letters in Microsoft Word and Apple Pages.

There's a high school teacher who teaches "Computer Science".  His course emphasizes drawing and painting in Photoshop, and animating things in Flash.  Yes, I know you can do a lot of programming in Flash.  No, they don't do very much programming in that course.

Teachers: Please, don't call typing and formatting a letter in a word processor "Computer Science". Please don't call a multimedia class "Computer Science".  It's confusing students into thinking "Computer Science" is what used to be called "Computer class".

If your class has a mix of subject matters, call it "Computer class", then within it you can have units on "Business Applications" or "Information Processing", "Multimedia", and maybe even a unit on "Computer Science" (which would only have programming and/or the more "math side" of computer science).

Otherwise, students will one day step foot in a real Computer Science class, be it in a high school with a real CS focused teacher, or in college or university.  And they're going to go "WTF, this isn't CS", when in fact they were duped in earlier grade school.

I've seen this happen before in a different subject: engineering.  Schools started offering "engineering" courses that were really just the old electro-technologies courses, which were closer to engineering-technician courses.  Imagine the surprise when some students who took those courses and got into university engineering then faced down the barrage of engineering math and physics courses.

Actually, who am I kidding.  It's already happening with Computer Science.  The term is meaningless among many junior high students.  Many of them think it's Photoshop, PowerPoint, or photography.

This is brand dilution.  And it's too late.

Worries over losing deep conceptual knowledge: Better teaching in any subject, part 4

• This is part of the "Better teaching in any subject" series:
• Part 1: Problems with modern inquiry based methods
• Part 2: Meaning is use
• Part 3: Facilitate learning through the inner game of meaning as uses
• Part 4: Worries over losing deep conceptual knowledge

A concern some may have with facilitating learning through the inner game of meaning-as-uses is that it seems to turn everything into decomposed techniques and skills, lacking in holistic, deep, conceptual, or otherwise "meaningful" knowledge.

A moment's thought should give you comfort that that's very far from the truth.

Imagine complaining that by breaking tennis up into the various forehand and backhand techniques, players will lose sight of the holistic meaningful concepts required to understanding tennis.

I'd imagine a lot of the worry comes about from not "seeing" the holistic concepts being taught or learned when seeing only the individual techniques being learned.  Traditionally, we'd see the worksheets for practicing factoring quadratics but never see the worksheets for learning what quadratics are conceptually, or what the meaning of factoring is.  From that, we might be led to believe that there must be something wrong with worksheets or with not teaching concepts and meanings (as if we could even directly teach concepts or meanings at all).

Facilitate learning through the inner game of meaning as uses: Better teaching in any subject, part 3

• This is part of the "Better teaching in any subject" series:
• Part 1: Problems with modern inquiry based methods
• Part 2: Meaning is use
• Part 3: Facilitate learning through the inner game of meaning as uses
• Part 4: Worries over losing deep conceptual knowledge (forthcoming)

The inner game of meaning: a lesson from tennis

A lesson from the Inner Game of Tennis (Gallwey) we might draw from is that consciously and intellectually solving the problem of "what is the instructor doing" is like a fool's errand. Because the actual problem the student is trying to solve is "how do I [the student] hit that tennis ball in that situation".  Solving the former problem may help with solving the latter, but there is no guarantee of effectiveness or efficiency.

Because of the unique cognitive and physical characteristics of each student, the solution to the actual problem is always unique anyway.  It always require each student to solve it anew.  The instructor can only point in a general direction, but the student has to go the final distance to arriving at a personalized solution.

If a student's energy is devoted to solving the problem of "what is the instructor doing", then the student will have little energy left for what is more important: solving the actual problem anew for themselves in a way that fits their own unique cognitive and physical characteristics.

How to facilitate learning proper meanings from proper uses

"Meaning is use" means that meaning comes from a variety of particular uses, and students need to look and see while teachers show the varieties of uses properly in order to learn the proper meanings from their proper uses.  But because every student brings with them a different set of prior learning and experiences, a way of conceptual thinking (and physical doing) that works for one student may not work for another.

Meaning is use: Better teaching in any subject, part 2

• This is part of the "Better teaching in any subject" series:
• Part 1: Problems with modern inquiry based methods
• Part 2: Meaning is use
• Part 3: Facilitate learning through the inner game of meaning as uses
  (forthcoming)
• Part 4: Worries over losing deep conceptual knowledge (forthcoming)

Meaning is use

Meaning as use is a philosophical concept of meaning from Wittgenstein:

"For a large class of cases of the employment of the word 'meaning' --- though not for all --- this way can be explained in this way: the meaning of a word is its use in the language" (Philosophical Investigations).

Many traditional and folk understanding of meaning explains meaning in terms of mental representations, or idealized objects in some (sometimes mathematical) objective space, etc. --- i.e. stuff in people's heads or in some Platonic ideal space that has no practical significance for teachers in the classroom.

So while we may not necessarily agree that meaning is philosophically just its use in the language, it's certainly practical to see it that way for teaching!  Because we can, as Wittgenstein urges, look and see the variety of cases in which a word is used, but we cannot look into the heads and minds of students --- and more importantly, nor can students look into the minds of teachers in learning what the teacher meant.

"So different is this new perspective that Wittgenstein repeats: 'Don't think, but look!' (PI 66); and such looking is done vis a vis particular cases, not generalizations." [1]

Meaning as discussed usually refers to meaning of words in a language, but math is no different.  Math is itself a natural language, with a grammar and semantics that's evolved in the mathematical community, used to talk about things and their relationships.  We need not look further than many science research papers wherein authors write mathematical notations and formulas, interweaved with English prose, to see how math is very much a language we can talk about things with.

If we accept that meaning (of words) comes from their use, then the meaning of a thing like a math formula is also built up from the use of it.  The proper meaning of a math formula doesn't come from the instructor explaining it, and it doesn't even come from students discussing and talking about it.  The meaning of a math formula comes from the proper use of it.

Problems with modern inquiry based methods: Better teaching in any subject, part 1

• This is part of the "Better teaching in any subject" series:
• Part 1: Problems with modern inquiry based methods
• Part 2: Meaning is use (forthcoming)
• Part 3: Facilitate learning through the inner game of meaning as uses
  (forthcoming)
• Part 4: Worries over losing deep conceptual knowledge (forthcoming)

Problems with experiential, discovery, inquiry, and constructivist learning and teaching

In education, teachers nowadays are often taught constructivism and other modern inquiry based teaching and learning methods.  Those teaching methods purport to help educators teach children in a way that helps the kids construct their own meaning of what they are to learn.  One of the central claims is that meaning is constructed through experiencing, and reflecting on those experiences, on the basis of concepts and meanings learned previously.

By "modern inquiry based" teaching and learning methods, I mean the constellation of academic philosophies and folk understandings of experiential, discovery, inquiry, and constructivist learning and teaching methods.

What's frustrating is that the core understandings in modern inquiry based teaching and learning methods are not so much as wrong, but are just not very helpful to teachers.  Not helpful because the core pedagogical ideas basically only tell teachers that kids must learn from experiences and reflection.  Since we're not privy to see or control all the stuff that happens in the kids' heads anyway, therefore all the philosophically interesting parts of constructivism have no practical significance in the classroom.

Reasons to learn math you'll never use

The film, "An Education", is about the experience of a teenaged girl (Jenny) growing up, meeting an older man, and making decisions regarding pursuing school versus marriage.  There is a passage near the end of the film that rather haunts me, especially the very last line (in bold below):

Jenny: Studying is hard and boring. Teaching is hard and boring. So, what you're telling me is to be bored, and then bored, and finally bored again, but this time for the rest of my life? This whole stupid country is bored! There's no life in it, or color, or fun! It's probably just as well the Russians are going to drop a nuclear bomb on us any day now. So my choice is to do something hard and boring, or to marry my... Jew, and go to Paris and Rome and listen to jazz, and read, and eat good food in nice restaurants, and have fun! It's not enough to educate us anymore Ms. Walters. You've got to tell us why you're doing it.

An Education

It strikes me that much of the frustration at all levels of education comes down to people having varying beliefs that are often contradictory, sometimes confused, and at other times missing altogether, about the purpose of education (and by that, I mean formal education in the school setting).

Some think education is for opening the mind of young people to the light of reason.  Some believe education is essentially career and job skills training.  Some think it is to "raise the kids" in the parenting sense, as though teachers are parents.  Many think it's more or less pointless, but essential as a child caring service while the children's parents are out making a living.  Some don't think about the question altogether, opting to do whatever the teachers say, as education is just traditionally "the thing to do".

The list of purposes ascribed to education by various people goes on...

Why push math education onto students?

Math education is important for many reasons, but I will focus on offering an answer from a societal economic policy level that stresses employment and the labour market.  I will refrain from mentioning all the other great reasons for having a great math education for children in this post.

This all came to my mind after reading Ramanathan's article in The Washington Post.  Ramanathan is a professor emeritus of math, statistics, and computer science, at the University of Illinois.  He contends that although a lot of effort and money has been put into making math seem essential to every child's and adult's daily life, it is not, as compared to history, politics, or music. Further, the article contends that no one should feel obligated to love math, any more than anyone is obligated to love grammar, or composition.  He contends that there is no evidence that all the money spent has actually helped student math achievement.

There really are a number of different issues that Ramanathan is attacking.  They include:

1. Should every person love math any more than grammar or whatever else that is perhaps boring and pedantic to some?
2. Has the money spent on improving math education been spent effectively?
3. Is math really as essential to every child's and adult's daily life as politics or history?
4. Given the answers to the previous questions, should extra effort or money be spent into improving math education and to improve students' math achievement?

The last question is, insofar as education is publicly funded (it is, more of less, in Canada. I am aware the writer is talking about the USA), a matter of public policy.  So we have to put our public policy hat on before continuing, because it makes a difference in how the question is answered.

The difference is this: when it comes to public policy, the answer to the first three questions are irrelevant to the fourth.

Here's why.

Suppose you're wrong. How would you know?

I'm asking this question in the scenario where you do not go out of your way to find out whether you're right or wrong. That is, based only on the regular experience and data that you would collect while doing nothing out of the ordinary, how would you know if you were wrong about what you ordinarily believe.

The point of the question is to highlight cases where you can't trust a belief.  I have, ordinarily, too many beliefs to be skeptical about and to prove, if I truly want certainty.  Fortunately, the world provides us with plenty of data through our ordinary experience, and it allows us to form pretty trustworthy beliefs, most of the time.

It's good to question at least some of our beliefs, though, especially in cases where you maybe shouldn't trust a belief, without first having gone out of your way to gather data to refute or verify it.

A simple example: suppose I believe I can walk through walls.  Well, how would I know I was wrong?  Without going out of my way, I can recall all the times I've accidentally backed into, or bumped into, a wall.  The ordinary course of experience provides me data to the contrary of that belief.  It's easy to be disabused of beliefs like being able to walk through walls or being able to fly.

Case 1

You may have heard of the "statistic" that 99 Out Of 100 Programmers Can’t Program. Maybe you believe it. Maybe you don't.  But now, just suppose it's wrong.  How would you know without going out of your way to gather data?

I would know because based on my experience, I can recall all the times I've met or talked to programmers who can program.  In fact, I might recall that the vast majority of programmers I know of can program just fine.  That's how I would know if the statistic is wrong.

Well, in fact, based on my experience, that is true of all the people I consider programmers of interest (so I'll exclude programmers who are just starting to learn, for example).  So without going out of my way, I would believe that statistic is wrong.

Notice that this is basically gathering data through a convenience sampling.  It's convenient because the sample has already been gathered through the course of ordinary experience!

Now that doesn't mean the statistic is wrong on the whole.  Perhaps a random sample would prove the statistic.  But I'll be lazy for a moment, and instead, try to understand the context where that original "statistic" was proposed.  That statistic was proposed by people doing hiring, but I have never had the experience of hiring someone.  So what might be a better explanation for both my experience-created convenience sample, and the proposed "statistic"?

A Little Knowledge is a Useless Thing

So the Autism-vaccine study has been retracted by the medical journal, The Lancet. It was the study that had led to the resurgence of measles in Europe by influencing parents to avoid the measles, mumps and rubella, or MMR, vaccine.

First, bravo on retracting that study. It's great to see irresponsible and dishonest research conduct meeting its just consequences.

The entire incident does, however, make me wonder whether it's a very bad thing for people, and non-scientists in particular, to believe everything that's published in a journal.

Book Review: "Hyperspace: A Scientific Odyssey Through Parallel Universes, Time Warps, and the 10th Dimension"

Wow, what can I say about this book? It inspired me to learn more science, specifically physics. Hyperspace: A Scientific Odyssey Through Parallel Universes, Time Warps, and the 10th Dimension has vivid analogies that I was able to, more or less, understand when I was about 11 years old.

The thought experiments Kaku talked about made me imagine the possibilities of the universe. It's a great book to get kids interested in science since it explains the intuitions without being stuck in the precise details.

Intuitions of what? Well, intuitions of space, time, and ultimately spacetime of more than three dimensions. It has nice thought experiments to help develop intuitions of what it's like to have more than three dimensions. Kaku also explains the intuitions around string theory, which is fascinating as well (even if it turns out that string theory is wrong, it's still fun to imagine the universe in a different way).

A great book, especially for kids. Highly recommended!

Open Recipe: Why Open Source Methodology in Science Matters

If data is the ingredients, then methodology is the recipe. Previously in Credibility of Science and the Freedom of Data, I raised the issue that even so-called raw data is often interpreted data, just as the temperature reading of a classic mercury-in-glass thermometer is really an interpretation of the height of the column of mercury. Modern scientific instruments wrap a lot of interpretations of the raw sensor feed up before presenting anything to the user as "raw" data.  But why does this even matter?

A key component that makes science so effective for discovering useful properties of the world is that experiments can, and are, repeated to ensure discoveries are in fact consistent properties of the world rather than the result of mere chance or accident. Being able to repeat some procedure to get back the same or similar result is a very important and useful feature of science. It is like having a recipe that claims to produce a great souffle: we'd have to verify the claim by repeatedly baking souffles to see if we repeatedly get good tasting souffle. We could, however, only do this if the entire recipe is open for us to follow and examine.

Science Losing Its Credit Over Money

In Tarnishing Science, I said "That scientists are human and are susceptible to ordinary persuasions like power, prestige, or money, isn't news."  What is news is just how widespread some of the problems are becoming, due to money, money, and more money.

Previously, I've offered my views on how scientific credibility could be enhanced through using open sourcing the scientific methodologies and allowing the free and open source distribution and use of the scientific data.

When it comes to the influence of money though, that's probably not enough.

Case in point, industry-sponsored medical science studies (see Med schools not responding to ghostwriting scandals). What can be done about the billions of dollars in biomedical research? All the money flowing into that research is a great thing, but even without maliciously cheating in the scientific process, the money can be targeted into research programs that are inherently biased in favour of the company bankrolling it. In Economics, this would be called voting with your wallet.

Re: "Call For Scientific Research Code To Be Released"

The Guardian is running this piece "If you're going to do good science, release the computer code too", about the need to have open source program code from scientific studies.  And Slashdot picked up on it too: Call For Scientific Research Code To Be Released.

It's good to see this view being recognized. I have, of course, expressed similar ideas before as in Credibility of Science and Open Source Methodology. Beyond open sourcing the methods, I also recommended the opening of the data sources as in Credibility of Science and the Freedom of Data, as a response to the recent Tarnishing of Science's respectability from the Climategate incident.

We need to recognize that this will continue to be a recurring issue in science as computing technology continues to revolutionize how all science is done.

"As the science scandals keep coming"...

The subtitle to The great global warming collapse (from The Globe and Mail) is:

As the science scandals keep coming, the air has gone out of the climate-change movement.

That is an example of the very point I made in Tarnishing Science:

It's concerning when I hear people mock or ridicule climate science research as "climate science" rather than as a "climate pseudo-science".  It's concerning because it suggests, to those who don't spend time looking carefully [2] at the situation, that it's not just climate science that has problems, it's that science — all of it — has problems.

Sure, The Globe and Mail article's hits are specifically delivered to climate science, but fairly or not, I'm afraid the lasting damage will be to the image of all sciences. It certainly will be that way for those who are, or who are leaning in the direction of, anti-scientific or anti-intellectual thinking.

Credibility of Science and Open Source Methodology

Having discussed the issue of the need for the freedom of data in Credibility of Science and the Freedom of Data given the slightly tarnished credibility of the sciences, in this post, I'll speak briefly on the issue of the methodology in science and why open sourcing it would help.

Here's what I mean for the methodology of an experiment to be open source: that is all the steps and instruments must be open to careful examination and testing. All the steps from the test tube to the data analysis must be open for examination to ensure validity. Traditionally, this hasn't been a problem at all. Beakers, test tubes, thermometers, etc, are physical objects inherently open to inspection, in case the instruments themselves added something into the materials, ingredients, or skew the data of the experiment. More and more, however, the modern instruments are becoming opaque to the scientists, especially as its insides are often controlled with computer and software technologies. Unfortunately, software is notoriously closed off and difficult to inspect.