Needed to work

• [ ] actually run the dark code // stachu
  • [ ] what is the structure that allows a user’s code to be run via Dark (1 canvas, repls, parse strings)
  • [ ] add user values to symtable
  • [ ] display results of running code
• [ ] Ensure analytics data isn’t lost
  • [ ] save analytics to disk?
  • [ ] update run-canvas-hack script to not clear the canvas when we redeploy?
  • [ ] figure out how to deal with DB schema changes between deploys?
• [x] use gpt4 or 3.5? We’ll use 3.5

Security

• [ ] (how) do we handle/prevent bad actors? i.e. “write me a program that does xyz terrible things”
  • [ ] have mechanism to keep track of usage/spending
  • [ ] have ability to turn it off
• [ ] add privacy policy to page
  • [ ] add draft to page //ocean
    • [ ] “we dont look at vars or output”
    • [ ] “we do look at user prompt and generated code”
  • [ ] improve privacy policy // paul
• [ ] disable tracing // stachu

Infra // paul

• [ ] have a deploy script
• [ ] BwdServer deployed to run this
  • [ ] hooked up to some DB? (neon? cockroach serverless?)
  • [ ] canvas hack the DB - script to do it
  • [ ] not deployed via CI
  • [ ] configure cloud run
  • [ ] add env vars for all services (eg launchdarkly) that are not the same as classic-dark
• [ ] experiment1.darklang.com
  • [ ] dns

UX

• [ ] dark branding/colors //ocean

• [ ] “run it” button //ocean

• [ ] “let me edit” button? //ocean

• [ ] show results somewhere //ocean

• [ ] what analytics are we getting and how do we get it //nico (stachu mentoring)

  • [ ] define “Analytics” DB in dark-editor canvas something like

  type AnalyticsAction =
    | CodeExecuted
    | ThumbsUp
    | ThumbsDown
    | CodeExecuted of hasInputVariables: bool
  
  [<DB("Analytics")>]
  type Analytics = 
    { SessionID: Uuid
      DarkPrompt: String
      UserPrompt: String
      GeneratedCode: String
      // TODO: Will this work? UserEnums probably don't work in 
      // user DBs at this point I think
      Action: AnalyticsAction }
  

  • [ ] add record to DB when code is (re)generated
  • [ ] add record to DB when thumbs up/down on the generated code
  • [ ] add record to DB when code is executed

  I think, from the above, we’ll have enough context gathered to learn things and we’ll be able to interpret things like “is the prompt changed?” implicitly

  Older notes:

  • [ ] what
    • [ ] is prompt changed? if so what is it
    • [ ] user prompt
    • [ ] generated code
    • [ ] did they have to edit the generated code
      • [ ] if so, original code and edited code
    • [ ] did they input variables
    • [ ] thumbs up/thumbs down
    • [ ] did they run it? Result type (eg strings vs derror)
  • [ ] how
    • [ ] save to db for now
    • [ ] schema/db in canvas
    • [ ] id for each refresh

• [ ] give us feedback! (box that emails us) //ocean