할 일

  • 생각없이 살지 말기, 핸드폰 그만 보기
  • be kind to me. 근데 그 전에 열심히 살자

Hallucination 알아보기

  • 논문 리스트
    • https://arxiv.org/pdf/2310.16045.pdf
    • https://arxiv.org/pdf/2311.03287.pdf
    • https://arxiv.org/pdf/2311.17911.pdf
    • https://arxiv.org/pdf/2311.16922.pdf
    • https://arxiv.org/pdf/2311.16839.pdf
    • https://arxiv.org/pdf/2311.13614.pdf
    • https://arxiv.org/pdf/2310.01779.pdf
    • https://arxiv.org/pdf/2309.04041.pdf
    • https://arxiv.org/pdf/2309.02301.pdf
    • https://arxiv.org/pdf/2308.15126.pdf
    • https://arxiv.org/pdf/2308.12714.pdf
    • https://arxiv.org/pdf/2308.06394.pdf
    • https://arxiv.org/pdf/2306.14565.pdf
    • https://arxiv.org/pdf/2305.10355.pdf
    • https://arxiv.org/pdf/2306.14565.pdf
    • https://arxiv.org/pdf/2311.03287.pdf
    • https://arxiv.org/pdf/2308.06394.pdf
  • 종류란?

instruct blip도 알아야지

어제 보던 issue 마저 보기

  • https://github.com/salesforce/LAVIS/issues/390
    • report된 것보다 성능이 낮아요.
  • https://github.com/salesforce/LAVIS/issues/389
  • https://github.com/salesforce/LAVIS/issues/388
    • instructblip의 script는 아직임(july10)
  • https://github.com/salesforce/LAVIS/issues/387
  • https://github.com/salesforce/LAVIS/issues/295
    • instructblip dataset이 없어요
  • https://github.com/salesforce/LAVIS/issues/365
  • https://github.com/salesforce/LAVIS/issues/297
    • 결과의 문자가 이상합니다.
  • https://github.com/salesforce/LAVIS/issues/370
    • 평가가 이상하면 평가 프롬프트를 바꿔보세요(논문 부록 E)
  • https://github.com/salesforce/LAVIS/issues/357
    • 긴 테스크 작업, multi round 대화
  • https://github.com/salesforce/LAVIS/issues/305
  • https://github.com/salesforce/LAVIS/issues/301
  • https://github.com/salesforce/LAVIS/issues/349
    • train config
  • https://github.com/salesforce/LAVIS/issues/302
    • script를 주세요!
  • https://github.com/salesforce/LAVIS/issues/292
    • 데이터셋은 오픈됐으니 instruct 변환해서 써라
  • https://github.com/salesforce/LAVIS/issues/342
    • 논문에서 설명한 implementation이 이상합니다
  • https://github.com/salesforce/LAVIS/issues/327
    • conf 파일 없나요?
  • https://github.com/salesforce/LAVIS/issues/313
    • instruct blip run
  • https://github.com/salesforce/LAVIS/issues/311
    • 짧고 반복적인 문장 생성

계획 세우기

  • Correction Task 추가

예상치 못한 문제

  • instruction들이 구현이 안되어 있다.
  • instruct blip은 그대로 쓴다고 해도 어떻게 해야되지?
    • https://github.com/salesforce/LAVIS/issues/139

일단

  • instruct blip 불러와서 test
  • instruct blip benchmark