CONFIDENTIAL INFORMATION
This document contains confidential and proprietary information intended solely for the use of
the individual or entity to whom it is disclosed.
Laurelin Sun Reasoning
Attempter Specifications
Project Overview
Task Attempt Overview
Task Attempt Workflow
Step 1: Evaluate and Review the problem
Step 2: Solve the prompt and note your thought processs
Step 3: Review the Ground Truth Final Answer
Step 4: Provide a Hint to guide the model
Hint Guidance for math problems
Appendix
Good & Bad Hint Examples
Helpful Links
Common Errors
Project Overview
Welcome to Laurelin Sun, a unique project that involves solving a challenging reasoning
problem and creating hints to guide a model to solve the problem on its own. The idea is to
gently nudge the model in the right direction without being too prescriptive, in order to see if the
model can get to the right answer with the right guidance.
Task Attempt Overview
This task is made up of the following 2 Steps:
Step 1: Evaluate and Review the problem (see Step 1 in Task Specifications)
A problem will be provided in every task.
If you are not able to solve it, you should skip the task until you find one
that you’re able to solve.
(Keep in mind that these are problems that have stumped the model, so they
are meant to be challenging.)
Step 2: Solve the Prompt and Note Your Thought Process (see Step 2 in Task
Specifications)
Write down your chain of thought.
Use the text box as a notepad to document your reasoning.
Capture every thought, even incorrect approaches.
Do not spend too much time on this, as it is not the main focus of the task.
Step 3: Review the Ground Truth Final Answer (GTFA) (see Step 3 in Task
Specifications)
If the GTFA provided is correct, select “Yes” and continue with the task
If the GTFA provided is incorrect, select “No” and write the correct GTFA
Remember that the GTFA should not include full sentences or
explanations.
Allowed formats:
Numbers
Intervals
Equations
Booleans
Sets of values
Vectors or matrices
Keep the same LaTeX format used in the prompt
Step 4: Provide a hint to guide the model to the correct answer (see Step 4 in Task
Specifications)
The hint should have enough guidance to direct the model on the right
thinking path, but does not need to spell out the right answer.
Think of this as tutoring someone - you want to provide a hint that gets them
on the right track without giving them the answer so they can learn on
their own.
Make sure the model reaches the correct final answer using your hint.
Watch Out: There is more than one way to render Latex in the task (using $...$ or \( … \))Across
the task PROMPT, GTFA AND HINT MUST HAVE THE SAME LATEX FORMAT!!!!!!
Task Attempt Workflow
Step 1: Evaluate and Review the problem
A problem will be provided in every task. If you are uncomfortable with it, you should skip until
you find one that you’re comfortable with.
Review and evaluate the problem. If you are not able to solve it, you should skip the task
until you find one that you’re able to solve. If you feel comfortable with the prompt,
please select if its solvable or not.
Step 2: Solve the prompt and note yout thought processs
Write down your chain of thought.
Use the text box as a notepad to document your reasoning.
Capture every thought, even incorrect approaches.
Accuracy of these notes is not graded; focus on transparency.
Do not spend too much time on this, as it is not the main focus of the task.
Do not worry much about LaTeX.
Step 3: Review the Ground Truth Final Answer
If the GTFA provided is correct, select “Yes” and continue with the task
If the GTFA provided is incorrect, select “No” and write the correct GTFA
Remember that the GTFA should not include full sentences or explanations.
Allowed formats:
Numbers
Intervals
Equations
Booleans
Sets of values
Vectors or matrices
Keep the same LaTeX format used in the prompt
The Ground Truth Final Answer you provide should be the expected answer to the prompt. Only
the solution should be recorded; any additional words should not be included.
Acceptable answers: 7; 2\pi; no solution; (0,4); x = 19, y = 0
Unacceptable answers: The answer is 8; Based on the analysis, x is 19 and y is 0; all
conclusive evidence supports that there is no possible solution to the problem of
x^2 = -2.
You should also write your Ground Truth Final Answer in LaTeX (LaTeX guide here), if needed.
There is a LaTeX Preview at the bottom to show you if your formatting renders correctly!
Step 4: Provide a Hint to guide the model
Write a hint that will show the model how to get on the right track. Think of the hint as general
guidelines or suggested approaches to solving the problem.
See Good & Bad Hint Examples in the appendix.
Remember: There is more than one way to render Latex in the task (using $...$ or \( … \))Across
the task PROMPT, GTFA AND HINT MUST HAVE THE SAME LATEX FORMAT!!!!!!
(Example: if the prompt uses $ .. $, you need to use that same symbol across
every step of the task.)
Hint Guidance for math problems
Note: The hint should not include the entire answer to the prompt.
Hint Writing Style
The hint should be written in natural language (including formulas if really needed) as
if you were a professor collaboratively trying to help a student solve a problem
that they haven't seen before.
The hint should not be vague.
The hint should be clear for someone that has a sound mathematical
background.
Hint Format
The hint should be a plan/strategy on how to solve the task.
The ideal format should be a short set of instructions to the model so that it can
answer the prompt.
What to Include in a Hint
The hint should include a set of key steps to solve the problem.
If a particular theorem is needed, the hint should mention the theorem.
If a particular equation/identity/formula is needed, the hint should mention the
equation/identity/formula.
Remember: There is more than one way to render Latex in the task (using $...$ or \( … \))Across
the task PROMPT, GTFA AND HINT MUST HAVE THE SAME LATEX FORMAT!!!!!!
(Example: if the prompt uses $ .. $, you need to use that same symbol across
every step of the task.)
There will also be an AutoCorrect with Feedback feature for the hint, to test for
spelling/grammar/logical reasoning.This is meant to be a helpful assistant as you write
your hint, however, use your judgment as the Chat Companion feature may not always
be correct.
You’ll need to click the Get Feedback button in order to click Submit Task (top right
corner).
Finally, you will need to make sure the model reaches the correct
final answer using your hint. If the model does not reach the
correct final answer, please rewrite/fix your hint.
Appendix
Good & Bad Hint Examples
Prompt Bad Hint Example Better Hint Example
Too vague Outlines steps for
May guide proof
models to Includes key
unnecessary theorems or
calculations ideas
Find all integers such Remember the modulo Find if there are any trivial
that operation solutions. Consider the equation
(mod 7). What possible values
(mod 7) can , have. A
technique such as reductio ad
infinitum may be needed.
Let be the field with Try to compute the structure Consider the non-zero
elements. Is every element of of the 18-th powers. elements of
a sum of 18-th powers? Or which form a cyclic
Show that the 18th-powers group of order 24
are a proper subfield of . Check that the non-zero
18-th powers are
also the non zero 6-
th powers, and that
these are a
subgroup of of
order 4.
Including the zero
element, there are 5
18-th powers, and
they form a subfield
of order 5.
Use the fact that a field
is closed under
addition.
Let be a square matrix that Generate the matrix with the The cube of a diagonal matrix is
is filled with zeros, except for numbers from to on its just the cubes of its diagonal
the coordinates where the row diagonal. Sort the rows of the elements, the trace of is just
number equals the column matrix according to the the sum of the cubes of
number. In those cells, the alphabetical order of the , regardless of the
numbers from to appear in names of the numbers. order. No need to sort the matrix.
alphabetical order (depending Calculate , and sum the Use the formula of the sum of
on how each number is written elements of its diagonal. the cubes.
in English, such as 1-3-2
because of 'one' - 'three' -
'two'). Find the trace of .
Helpful Links
Wolfram Alpha | You may use Wolfram Alpha to solve math problems by entering
equations or questions directly into its search bar. It will provide solutions,
graphs, and step-by-step explanations.
Desmos | Advanced graphing calculator, beneficial for various algebraic, geometric,
or calculus-based problems.
Calculator.Net | This site includes many helpful calculators, including quadratic
formulas, LCM, GCF, prime factorization, permutations, combinations, triangles,
volume, hex, and more.
GeoGebra | Similar to Desmos but with more functionality for geometry. You should
test this tool outside of task hours to gain familiarity, as there is a learning curve.
Common Errors
Do not write hints for the model that are too simple or brief.
