The Challenge: Create Accessible Resources
I work at a state university in the Commonwealth of Virginia. [Note that this document represents my personal experiences and opinions and is not meant to imply any endorsement by my university or the state—I'm just stating that for context.] Last year, the US federal government published final rulings on the ADA requirements for accessibility for web content on government websites, which includes state universities.
As a mathematician, I am used to posting documents in PDF form. Sometimes these are scans of handwritten solutions. Or they could be documents prepared using LaTeX. It is my understanding that both of these are problematic.
In this post, I want to explore dealing with handwritten solutions. The traditional work is just images. This would not meet the Perceivable or Understandable requirements. Is it possible for an instructor to start with their handwritten work and use current tools to convert it to an electronic representation that meets compliance?
Step 1: Create an example
I wanted to do a hands-on exploration. So I created a typical algebra exercise solution—solving a quadratic equation using factoring—and scanned it to a single page PDF. I also converted that image to a PNG file.
Step 2: Try AI
How well will a GPT AI model do at extracting my content. ChatGPT would not process the PDF version as it complained there was not text. When I attached the PNG image to my ChatGPT prompt, I was able to get something back successfully. I repeated the exercise on Microsoft Copilot, which is a resource that the university has licensed—I don't know details of what this license gives me.
ChatGPT
Here is the prompt I used for ChatGPT (GPT-4-turbo):
Convert this image to a markdown file, using latex math formulas for the math that appears in the document.
1. Solve \( 3x^2 - 5x - 2 = 0 \)Look for factors of \( (3)(-2) = -6 \) that add to -5.\[-6 + 1 = -5 \quad \text{and} \quad (-6)(1) = -6\]Rewrite \( 3x^2 - 5x - 2 = 3x^2 - 6x + 1x - 2 \) and factor by grouping:\[(3x^2 - 6x) + (x - 2) = 0 \\3x(x - 2) + (x - 2) = 0 \\(3x + 1)(x - 2) = 0\]Because this is now in the factored form \( AB = 0 \), we can solve each factor as an independent equation:\[3x + 1 = 0 \quad \Rightarrow \quad 3x = -1 \quad \Rightarrow \quad x = -\frac{1}{3}\]\[x - 2 = 0 \quad \Rightarrow \quad x = 2\]The solutions are \( x = -\frac{1}{3} \) and \( x = 2 \).
I think ChatGPT did quite a good job. I did notice that it made some editorial decisions. In the first step of the solution, it interpreted the two statements about the sum and products of the factor as a single display equation with the word "and" separated as text. That seems reasonable.
Then at the last major step, I solved each factor equal to zero. On paper, I did this in two columns, with each step a line below. ChatGPT decided to interpret that as two separate display statements (one for each column) using \Rightarrow as a separator between the steps. Again, I think this is a good choice.
Copilot
Here is the prompt I gave Copilot (GPT-4):
The attached image is a scan of a solution to a math exercise. Convert the image to a markdown file using LaTeX representations for each formula that appears in the document.
And here is the resulting markdown:
1. Solve \(3x^2 - 5x - 2 = 0\)Look for factors of \((3)(-2) = -6\) that add to \(-5\):- \(6 + (-1) = 5\)- \((-6)(1) = -6\)Rewrite \(3x^2 - 5x - 2 = 3x^2 - 6x + 1x - 2\) and factor by grouping:\[ (3x^2 - 6x) + (1x - 2) = 0 \]\[ (3x(x-2)) + (1(x-2)) = 0 \]\[ (3x+1)( x-2 )=0 \]Because this is now in the factored form \(AB=0\), we can solve each factor as an independent equation:\[\begin{cases}3x+1=0 \\x-2=0\end{cases}\]Solving these equations:\[\begin{cases}x=-\frac{1}{3} \\x=2\end{cases}\]The solutions are \( x=-\frac{1}{3} \) and \( x=2 \).
I immediately saw two problems with this translation.
Where ChatGPT took that pair of equations dealing with factors of -6 and interpreted it as a single display equation, Copilot decided to reinterpret my work. The equation that should have said "-6+1 = -5" as instead became a bullet item with a different equation "6+(-1)=5".
Also, when solving the individual factors as equations, Copilot stated the original equations and then stated the answers. It skips the intermediate step that I had given "3x=-1" when solving "3x+1=0".
Conclusions
I think using GPT models to convert the images seems fairly promising, but it is clear that it will take proofreading to make sure that the AI didn't introduce changes to the solutions. I am also somewhat concerned about the tendency for the AI to try to restructure how the flow is presented.
Step 3: Convert to PreTeXt
Markdown alone is not going to be accessible. But it should be possible to generate an HTML version of the document using the PreTeXt document system. Since I already had ChatGPT create reasonable markdown, I asked it to finish the process and convert the markdown to PreTeXt.
It did okay, but the LaTeX formatted equations were left in LaTeX style using \( and \) or \[ and \]. So I had to repeat my prompt specifying that inline formulas should use <m> tags while display formulas should use <me> tags. This wasn't quite right, as I next needed to add that multi-line equations should use <md> tags with <mrow> tags around equation separate line.
It gave me back a PreTeXt markup of an exercise with a solution.
<exercise><title>Solve a Quadratic Equation by Factoring</title><statement><p>Solve the equation <m>3x^2 - 5x - 2 = 0</m>.</p></statement><solution><p>Look for factors of <m>(3)(-2) = -6</m> that add to <m>-5</m>.</p><ul><li><m>-6 + 1 = -5</m></li><li><m>(-6)(1) = -6</m></li></ul><p>Rewrite <m>3x^2 - 5x - 2</m> as <m>3x^2 - 6x + x - 2</m>, and factor by grouping:</p><md><mrow>(3x^2 - 6x) + (x - 2) = 0</mrow><mrow>3x(x - 2) + (x - 2) = 0</mrow><mrow>(3x + 1)(x - 2) = 0</mrow></md><p>Because this is now in the factored form <m>AB = 0</m>, we can solve each factor as an independent equation:</p><ul><li><me>3x + 1 = 0</me><me>3x = -1</me><me>x = -\frac{1}{3}</me></li><li><me>x - 2 = 0</me><me>x = 2</me></li></ul><p>The solutions are <m>x = -\frac{1}{3}</m> and <m>x = 2</m>.</p></solution></exercise>
This was a pretty good conversion. Notice, however, that the solutions for the separate factors is no longer a display equation, but is a series of display equations in a bullet list. Curiously, this was not done using the <md> tag.
When I asked ChatGPT to finish the work by wrapping the content in <handout>, it generated its own title and introduction to the exercise.
<handout xml:id="quadratic-factoring-example" xmlns="http://pretextbook.org"><title>Solving a Quadratic Equation by Factoring</title><introduction><p>This example demonstrates how to solve a quadratic equation using factoring and grouping techniques.</p></introduction><exercise><title>Solve a Quadratic Equation by Factoring</title><statement><p>Solve the equation <m>3x^2 - 5x - 2 = 0</m>.</p></statement><solution><p>Look for factors of <m>(3)(-2) = -6</m> that add to <m>-5</m>.</p><ul><li><m>-6 + 1 = -5</m></li><li><m>(-6)(1) = -6</m></li></ul><p>Rewrite <m>3x^2 - 5x - 2</m> as <m>3x^2 - 6x + x - 2</m>, and factor by grouping:</p><md><mrow>(3x^2 - 6x) + (x - 2) = 0</mrow><mrow>3x(x - 2) + (x - 2) = 0</mrow><mrow>(3x + 1)(x - 2) = 0</mrow></md><p>Because this is now in the factored form <m>AB = 0</m>, we can solve each factor as an independent equation:</p><ul><li><me>3x + 1 = 0</me><me>3x = -1</me><me>x = -\frac{1}{3}</me></li><li><me>x - 2 = 0</me><me>x = 2</me></li></ul><p>The solutions are <m>x = -\frac{1}{3}</m> and <m>x = 2</m>.</p></solution></exercise></handout>
Will it compile?
No. PreTeXt requires a project. This should be embedded within a larger project. The header of the XML should not have the language name space. Also, the "\frac" statements were converted so that "\f" was interpreted as a special character. Once I created a place-holder project and fixed the error with "\frac"
Here is a screenshot of the final result. This is listed as "Chapter 3" because I put this in a default PreTeXt "course" project under a chapter for solutions.
I am now wondering how this workflow could be simplified and made more reliable.
Do you have other ideas? Leave them in the comments.
