Skip to content

create new thoughtsource version #133

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
KonstantinHebenstreit merged 250 commits into from
Jul 11, 2023
Merged

create new thoughtsource version #133

KonstantinHebenstreit merged 250 commits into from
Jul 11, 2023

Conversation

@KonstantinHebenstreit
Copy link
Collaborator

@KonstantinHebenstreit KonstantinHebenstreit commented Jul 11, 2023

update thoughtsource to new version. this includes:

  • option for creating chained commands
  • added new datasets: MMLU (six medical subsets) and and open ended question version of MedQA
  • created thoughtsource_33 data collection, including examples from six dataset, summing to ~200 examples for fast testing
  • added chat option of gpt
  • added filtering functions for already created chain-of-thoughts

Konstantin Hebenstreit and others added 30 commits March 10, 2023 20:06
answer extraction trigger adaptive to question
84a4062
select works on multiple splits
5562b9b
include openai_chat
46a74e9
include control if api_service is available
763be78
update packages for openai_chat
43d638d
include "True","False" as choices in strategy_qa, necessary for annot…
3c95cf2 
…ator
corrected strategy_qa choices in thoughtsource_100
6c6afeb
delete thoughtsource_1_empty since it can now easily be created
5fa82b2
fix adaptive config to be called on every new dataset
9a53c4f
add item
e6e70cd
chat gpt notebook and datasets
aa8ac96
include chat-gpt in ts_100 data
35e3ae6
quick fix for chat gpt model
636b0ba
@elmaestrobert
mapping function for metareasoning
0713510
@elmaestrobert
Key mistake
3ea644f
@elmaestrobert
langchain self-reflect first part
fb466d5
@elmaestrobert
manipulating collection works with json bypass
c6d0bc9
@elmaestrobert
Update create_metadataset.ipynb
ab23b3a
@elmaestrobert
dataset manipulation and simple langchain chain
8b474cf
@elmaestrobert
Further code for reflection
a578f6c
@elmaestrobert
latest
b68126f
@elmaestrobert
structure of new generate function
e3c9d64
@elmaestrobert
v1 new generate function for simple cot generation
61b8288
@elmaestrobert
generate_extract v2
0b3348a
@elmaestrobert
data_map_error added
1387b37
@elmaestrobert
added explanation
8d8c5cd
@elmaestrobert
draft reflection_chain
9eb6dad
@elmaestrobert
General structure stands, small bugs yet to fix
7bd0485
@elmaestrobert
Code ready for experimentation
8513e5a
@elmaestrobert
start experimenting
c622aeb
elmaestrobert and others added 29 commits May 7, 2023 10:51
@elmaestrobert
tests
f69dd8e
@elmaestrobert
error_testing
bf4501d
@elmaestrobert
recache problem in 'flexible_langchains'
5c4d313
@elmaestrobert
generated example error in flexible langchains notebook
01ab854
@elmaestrobert
update
8b4fcc6
@elmaestrobert
update before merging in dev with answer_from_choices
8296727
@elmaestrobert
Merge branch 'dev' into chain_experiments
6e0dfb7
@elmaestrobert
build local model experiment, Collection error
8fb4e60
@elmaestrobert
cache problem
a5d2c24
@elmaestrobert
Revert "build local model experiment, Collection error"
47c786a 
This reverts commit 8fb4e60.
@elmaestrobert
soft revert, new error
51b739e
@elmaestrobert
Merge branch 'dev' of https://github.com/OpenBioLink/ThoughtSource in…
23e3bed 
…to dev
@elmaestrobert
answer_from_choices_fix
84c80d6
@elmaestrobert
all function now use data.map
199dad6
updates for students course
33ddf6c
@elmaestrobert
Merge branch 'dev' into flexible_chain_notebook
6aa02ab
@elmaestrobert
student course intermediate
d063183
@elmaestrobert
students_course with evaluation
12ea9c0
@elmaestrobert
re-ran notebook
8c64798
@elmaestrobert
Delete generate_chain_documentation
d0b9d3d
@elmaestrobert
update flexible langchain notebook
8becb53
correct the 'title' option in evaluation
536a6f6
correct one bool test option
0c91f49
fixing init file
948292f
fix loading datasets: restructuring mmlu files
2c0b81e
deactivate dataset test for source view
1c2ce30
readd dataset_analysis file
4293d31
include new paper and datasets into readme
414fec4
moving and sorting internal notebooks
3b31bd8
@KonstantinHebenstreit KonstantinHebenstreit merged commit 733f262 into main Jul 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@KonstantinHebenstreit @elmaestrobert @louispk @AsimO03