Commit rewriting application

.gitignore

@@ -0,0 +1,282 @@
+# Created by https://www.toptal.com/developers/gitignore/api/pycharm+all,venv,python
+# Edit at https://www.toptal.com/developers/gitignore?templates=pycharm+all,venv,python
+
+### PyCharm+all ###
+# Covers JetBrains IDEs: IntelliJ, RubyMine, PhpStorm, AppCode, PyCharm, CLion, Android Studio, WebStorm and Rider
+# Reference: https://intellij-support.jetbrains.com/hc/en-us/articles/206544839
+
+# User-specific stuff
+.idea/**/workspace.xml
+.idea/**/tasks.xml
+.idea/**/usage.statistics.xml
+.idea/**/dictionaries
+.idea/**/shelf
+
+# AWS User-specific
+.idea/**/aws.xml
+
+# Generated files
+.idea/**/contentModel.xml
+
+# Sensitive or high-churn files
+.idea/**/dataSources/
+.idea/**/dataSources.ids
+.idea/**/dataSources.local.xml
+.idea/**/sqlDataSources.xml
+.idea/**/dynamic.xml
+.idea/**/uiDesigner.xml
+.idea/**/dbnavigator.xml
+
+# Gradle
+.idea/**/gradle.xml
+.idea/**/libraries
+
+# Gradle and Maven with auto-import
+# When using Gradle or Maven with auto-import, you should exclude module files,
+# since they will be recreated, and may cause churn.  Uncomment if using
+# auto-import.
+# .idea/artifacts
+# .idea/compiler.xml
+# .idea/jarRepositories.xml
+# .idea/modules.xml
+# .idea/*.iml
+# .idea/modules
+# *.iml
+# *.ipr
+
+# CMake
+cmake-build-*/
+
+# Mongo Explorer plugin
+.idea/**/mongoSettings.xml
+
+# File-based project format
+*.iws
+
+# IntelliJ
+out/
+
+# mpeltonen/sbt-idea plugin
+.idea_modules/
+
+# JIRA plugin
+atlassian-ide-plugin.xml
+
+# Cursive Clojure plugin
+.idea/replstate.xml
+
+# SonarLint plugin
+.idea/sonarlint/
+
+# Crashlytics plugin (for Android Studio and IntelliJ)
+com_crashlytics_export_strings.xml
+crashlytics.properties
+crashlytics-build.properties
+fabric.properties
+
+# Editor-based Rest Client
+.idea/httpRequests
+
+# Android studio 3.1+ serialized cache file
+.idea/caches/build_file_checksums.ser
+
+### PyCharm+all Patch ###
+# Ignore everything but code style settings and run configurations
+# that are supposed to be shared within teams.
+
+.idea/*
+
+!.idea/codeStyles
+!.idea/runConfigurations
+
+### Python ###
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+# .python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# poetry
+#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
+#poetry.lock
+
+# pdm
+#   Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
+#pdm.lock
+#   pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
+#   in version control.
+#   https://pdm.fming.dev/#use-with-ide
+.pdm.toml
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+# PyCharm
+#  JetBrains specific template is maintained in a separate JetBrains.gitignore that can
+#  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
+#  and can be added to the global gitignore or merged into this file.  For a more nuclear
+#  option (not recommended) you can uncomment the following to ignore the entire idea folder.
+#.idea/
+
+### Python Patch ###
+# Poetry local configuration file - https://python-poetry.org/docs/configuration/#local-configuration
+poetry.toml
+
+# ruff
+.ruff_cache/
+
+# LSP config files
+pyrightconfig.json
+
+### venv ###
+# Virtualenv
+# http://iamzed.com/2009/05/07/a-primer-on-virtualenv/
+[Bb]in
+[Ii]nclude
+[Ll]ib
+[Ll]ib64
+[Ll]ocal
+[Ss]cripts
+pyvenv.cfg
+pip-selfcheck.json
+
+# End of https://www.toptal.com/developers/gitignore/api/pycharm+all,venv,python
+
+.idea
+
+cache
+feedback
+flagged

README.md

@@ -0,0 +1,6 @@
+---
+title: commit-rewriting
+app_file: app.py
+sdk: gradio
+sdk_version: 4.15.0
+---

app.py

@@ -0,0 +1,201 @@
+import os
+import random
+import uuid
+from datetime import datetime
+from difflib import ndiff
+
+import gradio as gr
+
+from data_loader import load_data
+
+HF_TOKEN = os.environ.get('HF_REWRITING_TOKEN')
+HF_DATASET = os.environ.get('HF_REWRITING_DATASET')
+N_QUESTIONS = 5
+
+data = load_data()
+
+n_samples = len(data)
+
+saver = gr.HuggingFaceDatasetSaver(HF_TOKEN, HF_DATASET, private=True)
+
+
+def convert_diff_to_unified(diff):
+    result = "\n".join(
+        [
+            f'--- {modified_file["old_path"]}\n'
+            f'+++ {modified_file["new_path"]}\n'
+            f'{modified_file["diff"]}'
+            for modified_file in diff
+        ]
+    )
+
+    return result
+
+
+def get_diff2html_view(raw_diff):
+    html = f"""
+    <div style='width:100%; height:1400px; overflow:auto; position: relative'>
+        <div id='diff-raw' hidden>{raw_diff}</div> 
+        <div class="d2h-view-wrapper">
+            <div id='diff-view'></div>
+        </div>
+    </div>
+    """
+
+    return html
+
+
+def get_github_link_md(repo, hash):
+    return f'[See the commit on Github](https://github.com/{repo}/commit/{hash})'
+
+
+def char_diff_obj(change_type, pos, character, timestamp):
+    return {"type": change_type, "pos": pos, "char": character, "timestamp": timestamp}
+
+
+def update_commit_view(sample_ind):
+    if sample_ind >= n_samples:
+        return None
+
+    record = data[sample_ind]
+
+    diff_view = get_diff2html_view(convert_diff_to_unified(record['mods']))
+
+    repo_val = record['repo']
+    hash_val = record['hash']
+    github_link_md = get_github_link_md(repo_val, hash_val)
+
+    diff_loaded_timestamp = datetime.now().isoformat()
+
+    commit_message = record['prediction']
+    commit_message_start = commit_message
+    commit_message_prev = commit_message
+    commit_message_history = []
+
+    return (
+        github_link_md, diff_view, repo_val, hash_val, diff_loaded_timestamp,
+        commit_message_start, commit_message, commit_message_prev, commit_message_history)
+
+
+def next_sample(current_sample_ind, shuffled_idx):
+    if current_sample_ind == n_samples:
+        return None
+
+    current_sample_ind += 1
+    updated_view = update_commit_view(shuffled_idx[current_sample_ind])
+    return (current_sample_ind,) + updated_view
+
+
+with open("head.html") as head_file:
+    head_html = head_file.read()
+
+with gr.Blocks(theme=gr.themes.Soft(), head=head_html, css="style_overrides.css") as application:
+    repo_val = gr.Textbox(interactive=False, label='repo', visible=False)
+    hash_val = gr.Textbox(interactive=False, label='hash', visible=False)
+    shuffled_idx_val = gr.JSON(visible=False)
+
+    with gr.Row():
+        with gr.Accordion("Help"):
+            with open("survey_guide.md") as content_file:
+                gr.Markdown(content_file.read())
+
+    with gr.Row():
+        current_sample_sld = gr.Slider(minimum=0, maximum=n_samples, step=1,
+                                       value=0,
+                                       interactive=False,
+                                       label='sample_ind',
+                                       info=f"Samples labeled/skipped (out of {n_samples})",
+                                       show_label=False,
+                                       container=False,
+                                       scale=5)
+        with gr.Column(scale=1):
+            skip_btn = gr.Button("Skip the current sample")
+    with gr.Row():
+        with gr.Column(scale=2):
+            github_link = gr.Markdown()
+            diff_view = gr.HTML()
+        with gr.Column(scale=1):
+            commit_msg_start = gr.TextArea(label="commit_msg_start", visible=False)
+            commit_msg = gr.TextArea(label="commit_msg_end", show_label=False,
+                                     info="Commit message (can be scrollable)")
+            commit_msg_prev = gr.TextArea(visible=False)
+            commit_msg_history = gr.JSON(label="commit_msg_history", visible=False)
+
+            submit_btn = gr.Button("Submit")
+
+            session_val = gr.Textbox(info='Session', interactive=False, container=True, show_label=False,
+                                     label='session')
+
+            with gr.Row(visible=False):
+                sample_loaded_timestamp = gr.Textbox(info="Sample loaded", label='loaded_ts', interactive=False,
+                                                     container=True, show_label=False)
+                now_timestamp = gr.Textbox(info="Current time",
+                                           interactive=False, container=True, show_label=False,
+                                           value=lambda: datetime.now().isoformat(), every=1.0,
+                                           label='submitted_ts')
+
+    commit_view = [
+        github_link,
+        diff_view,
+        repo_val,
+        hash_val,
+        sample_loaded_timestamp,
+        commit_msg_start,
+        commit_msg,
+        commit_msg_prev,
+        commit_msg_history
+    ]
+
+    feedback_metadata = [
+        session_val,
+        repo_val,
+        hash_val,
+        sample_loaded_timestamp,
+        now_timestamp
+    ]
+
+    feedback_form = [commit_msg_start, commit_msg, commit_msg_history]
+
+    saver.setup([current_sample_sld] + feedback_metadata + feedback_form, "feedback")
+
+    skip_btn.click(next_sample, inputs=[current_sample_sld, shuffled_idx_val],
+                   outputs=[current_sample_sld] + commit_view)
+
+
+    def submit(current_sample, shuffled_idx, *args):
+        saver.flag((current_sample,) + args)
+        return next_sample(current_sample, shuffled_idx)
+
+
+    submit_btn.click(
+        submit,
+        inputs=[current_sample_sld, shuffled_idx_val] + feedback_metadata + feedback_form,
+        outputs=[current_sample_sld] + commit_view
+    )
+
+
+    def on_commit_msg_changed(message, prev_message, history, timestamp):
+        for i, s in enumerate(ndiff(prev_message, message)):
+            diff = char_diff_obj(s[0], i, s[-1], timestamp)
+            if diff['type'] in ('+', '-'):
+                history.append(diff)
+        return message, history
+
+
+    commit_msg.change(on_commit_msg_changed, inputs=[commit_msg, commit_msg_prev, commit_msg_history,
+                                                     now_timestamp],
+                      outputs=[commit_msg_prev, commit_msg_history])
+
+
+    def init_session(current_sample):
+        session = str(uuid.uuid4())
+        shuffled_idx = list(range(n_samples))
+        random.shuffle(shuffled_idx)
+        return (session, shuffled_idx) + update_commit_view(shuffled_idx[current_sample])
+
+
+    application.load(init_session,
+                     inputs=[current_sample_sld],
+                     outputs=[session_val, shuffled_idx_val] + commit_view, )
+
+application.launch()

data_loader.py

@@ -0,0 +1,22 @@
+from datasets import load_dataset
+
+MODEL = 'cmg_gpt_4_0613'
+CACHE_DIR = 'cache'
+
+
+def load_data():
+    dataset = load_dataset("JetBrains-Research/lca-cmg",
+                           "commitchronicle-py-long",
+                           split="test",
+                           cache_dir=CACHE_DIR).to_pandas().set_index(['hash', 'repo']).rename(
+        columns={'message': 'reference'})
+
+    model_dataset = load_dataset("JetBrains-Research/lca-results",
+                                 MODEL,
+                                 split="test",
+                                 cache_dir=CACHE_DIR).to_pandas().set_index(['hash', 'repo'])[["prediction"]]
+
+    model_dataset = model_dataset[~model_dataset.index.duplicated(keep='first')]
+    dataset = dataset.join(other=model_dataset)
+
+    return dataset.reset_index().to_dict('records')

head.html

@@ -0,0 +1,31 @@
+<!-- Javascripts -->
+<script type="text/javascript" src="https://cdn.jsdelivr.net/npm/diff2html/bundles/js/diff2html-ui.min.js"></script>
+
+<script>
+    var oldDiff = "";
+
+    function updateDiffView() {
+        try {
+            var diff = document.getElementById('diff-raw').innerHTML;
+            if (!diff || diff === oldDiff) {
+                return;
+            }
+            oldDiff = diff;
+            var targetElement = document.getElementById('diff-view');
+            var configuration = {
+                drawFileList: true,
+                matching: 'lines',
+                highlight: true,
+                colorScheme: 'light'
+            };
+            var diff2htmlUi = new Diff2HtmlUI(targetElement, diff, configuration);
+            diff2htmlUi.draw();
+        } catch (error) {
+
+        }
+    }
+
+    setInterval(function () {
+        updateDiffView();
+    }, 500);
+</script>

requirements.txt

@@ -0,0 +1,79 @@
+aiofiles==23.2.1
+aiohttp==3.9.1
+aiosignal==1.3.1
+altair==5.2.0
+annotated-types==0.6.0
+anyio==4.2.0
+async-timeout==4.0.3
+attrs==23.2.0
+certifi==2023.11.17
+charset-normalizer==3.3.2
+click==8.1.7
+colorama==0.4.6
+contourpy==1.2.0
+cycler==0.12.1
+datasets==2.16.1
+dill==0.3.7
+exceptiongroup==1.2.0
+fastapi==0.109.0
+ffmpy==0.3.1
+filelock==3.13.1
+fonttools==4.47.2
+frozenlist==1.4.1
+fsspec==2023.10.0
+gradio==4.15.0
+gradio_client==0.8.1
+h11==0.14.0
+httpcore==1.0.2
+httpx==0.26.0
+huggingface-hub==0.20.3
+idna==3.6
+importlib-resources==6.1.1
+Jinja2==3.1.3
+jsonschema==4.21.1
+jsonschema-specifications==2023.12.1
+kiwisolver==1.4.5
+markdown-it-py==3.0.0
+MarkupSafe==2.1.4
+matplotlib==3.8.2
+mdurl==0.1.2
+multidict==6.0.4
+multiprocess==0.70.15
+numpy==1.26.3
+orjson==3.9.12
+packaging==23.2
+pandas==2.2.0
+pillow==10.2.0
+pyarrow==15.0.0
+pyarrow-hotfix==0.6
+pydantic==2.5.3
+pydantic_core==2.14.6
+pydub==0.25.1
+Pygments==2.17.2
+pyparsing==3.1.1
+python-dateutil==2.8.2
+python-multipart==0.0.6
+pytz==2023.3.post1
+PyYAML==6.0.1
+referencing==0.32.1
+requests==2.31.0
+rich==13.7.0
+rpds-py==0.17.1
+ruff==0.1.14
+semantic-version==2.10.0
+shellingham==1.5.4
+six==1.16.0
+sniffio==1.3.0
+starlette==0.35.1
+tomlkit==0.12.0
+toolz==0.12.0
+tqdm==4.66.1
+typer==0.9.0
+typing_extensions==4.9.0
+tzdata==2023.4
+urllib3==2.1.0
+uvicorn==0.27.0
+websockets==11.0.3
+xxhash==3.4.1
+yarl==1.9.4
+zipp==3.17.0

style_overrides.css

@@ -0,0 +1,905 @@
+:host, :root {
+    --d2h-bg-color: #fff;
+    --d2h-border-color: #ddd;
+    --d2h-dim-color: rgba(0, 0, 0, .3);
+    --d2h-line-border-color: #eee;
+    --d2h-file-header-bg-color: #f7f7f7;
+    --d2h-file-header-border-color: #d8d8d8;
+    --d2h-empty-placeholder-bg-color: #f1f1f1;
+    --d2h-empty-placeholder-border-color: #e1e1e1;
+    --d2h-selected-color: #c8e1ff;
+    --d2h-ins-bg-color: #dfd;
+    --d2h-ins-border-color: #b4e2b4;
+    --d2h-ins-highlight-bg-color: #97f295;
+    --d2h-ins-label-color: #399839;
+    --d2h-del-bg-color: #fee8e9;
+    --d2h-del-border-color: #e9aeae;
+    --d2h-del-highlight-bg-color: #ffb6ba;
+    --d2h-del-label-color: #c33;
+    --d2h-change-del-color: #fdf2d0;
+    --d2h-change-ins-color: #ded;
+    --d2h-info-bg-color: #f8fafd;
+    --d2h-info-border-color: #d5e4f2;
+    --d2h-change-label-color: #d0b44c;
+    --d2h-moved-label-color: #3572b0;
+    --d2h-dark-color: #e6edf3;
+    --d2h-dark-bg-color: #0d1117;
+    --d2h-dark-border-color: #30363d;
+    --d2h-dark-dim-color: #6e7681;
+    --d2h-dark-line-border-color: #21262d;
+    --d2h-dark-file-header-bg-color: #161b22;
+    --d2h-dark-file-header-border-color: #30363d;
+    --d2h-dark-empty-placeholder-bg-color: hsla(215, 8%, 47%, .1);
+    --d2h-dark-empty-placeholder-border-color: #30363d;
+    --d2h-dark-selected-color: rgba(56, 139, 253, .1);
+    --d2h-dark-ins-bg-color: rgba(46, 160, 67, .15);
+    --d2h-dark-ins-border-color: rgba(46, 160, 67, .4);
+    --d2h-dark-ins-highlight-bg-color: rgba(46, 160, 67, .4);
+    --d2h-dark-ins-label-color: #3fb950;
+    --d2h-dark-del-bg-color: rgba(248, 81, 73, .1);
+    --d2h-dark-del-border-color: rgba(248, 81, 73, .4);
+    --d2h-dark-del-highlight-bg-color: rgba(248, 81, 73, .4);
+    --d2h-dark-del-label-color: #f85149;
+    --d2h-dark-change-del-color: rgba(210, 153, 34, .2);
+    --d2h-dark-change-ins-color: rgba(46, 160, 67, .25);
+    --d2h-dark-info-bg-color: rgba(56, 139, 253, .1);
+    --d2h-dark-info-border-color: rgba(56, 139, 253, .4);
+    --d2h-dark-change-label-color: #d29922;
+    --d2h-dark-moved-label-color: #3572b0
+}
+
+.gradio-container .main .d2h-view-wrapper {
+    * {
+        all: revert;
+    }
+
+    #diff-view {
+        pre code.hljs {
+            display: block;
+            overflow-x: auto;
+            padding: 1em
+        }
+
+        code.hljs {
+            padding: 3px 5px
+        }
+
+
+        /*!
+      Theme: GitHub
+      Description: Light theme as seen on github.com
+      Author: github.com
+      Maintainer: @Hirse
+      Updated: 2021-05-15
+
+      Outdated base version: https://github.com/primer/github-syntax-light
+      Current colors taken from GitHub's CSS
+    */
+
+        .hljs {
+            color: #24292e;
+            background: #fff
+        }
+
+        .hljs-doctag, .hljs-keyword, .hljs-meta .hljs-keyword, .hljs-template-tag, .hljs-template-variable, .hljs-type, .hljs-variable.language_ {
+            color: #d73a49
+        }
+
+        .hljs-title, .hljs-title.class_, .hljs-title.class_.inherited__, .hljs-title.function_ {
+            color: #6f42c1
+        }
+
+        .hljs-attr, .hljs-attribute, .hljs-literal, .hljs-meta, .hljs-number, .hljs-operator, .hljs-selector-attr, .hljs-selector-class, .hljs-selector-id, .hljs-variable {
+            color: #005cc5
+        }
+
+        .hljs-meta .hljs-string, .hljs-regexp, .hljs-string {
+            color: #032f62
+        }
+
+        .hljs-built_in, .hljs-symbol {
+            color: #e36209
+        }
+
+        .hljs-code, .hljs-comment, .hljs-formula {
+            color: #6a737d
+        }
+
+        .hljs-name, .hljs-quote, .hljs-selector-pseudo, .hljs-selector-tag {
+            color: #22863a
+        }
+
+        .hljs-subst {
+            color: #24292e
+        }
+
+        .hljs-section {
+            color: #005cc5;
+            font-weight: 700
+        }
+
+        .hljs-bullet {
+            color: #735c0f
+        }
+
+        .hljs-emphasis {
+            color: #24292e;
+            font-style: italic
+        }
+
+        .hljs-strong {
+            color: #24292e;
+            font-weight: 700
+        }
+
+        .hljs-addition {
+            color: #22863a;
+            background-color: #f0fff4
+        }
+
+        .hljs-deletion {
+            color: #b31d28;
+            background-color: #ffeef0
+        }
+
+
+        .d2h-wrapper {
+            text-align: left
+        }
+
+        .d2h-file-header {
+            background-color: #f7f7f7;
+            background-color: var(--d2h-file-header-bg-color);
+            border-bottom: 1px solid #d8d8d8;
+            border-bottom: 1px solid var(--d2h-file-header-border-color);
+            display: -webkit-box;
+            display: -ms-flexbox;
+            display: flex;
+            font-family: Source Sans Pro, Helvetica Neue, Helvetica, Arial, sans-serif;
+            height: 35px;
+            padding: 5px 10px
+        }
+
+        .d2h-file-header.d2h-sticky-header {
+            position: sticky;
+            top: 0;
+            z-index: 1
+        }
+
+        .d2h-file-stats {
+            display: -webkit-box;
+            display: -ms-flexbox;
+            display: flex;
+            font-size: 14px;
+            margin-left: auto
+        }
+
+        .d2h-lines-added {
+            border: 1px solid #b4e2b4;
+            border: 1px solid var(--d2h-ins-border-color);
+            border-radius: 5px 0 0 5px;
+            color: #399839;
+            color: var(--d2h-ins-label-color);
+            padding: 2px;
+            text-align: right;
+            vertical-align: middle
+        }
+
+        .d2h-lines-deleted {
+            border: 1px solid #e9aeae;
+            border: 1px solid var(--d2h-del-border-color);
+            border-radius: 0 5px 5px 0;
+            color: #c33;
+            color: var(--d2h-del-label-color);
+            margin-left: 1px;
+            padding: 2px;
+            text-align: left;
+            vertical-align: middle
+        }
+
+        .d2h-file-name-wrapper {
+            display: -webkit-box;
+            display: -ms-flexbox;
+            display: flex;
+            -webkit-box-align: center;
+            -ms-flex-align: center;
+            align-items: center;
+            font-size: 15px;
+            width: 100%
+        }
+
+        .d2h-file-name {
+            overflow-x: hidden;
+            text-overflow: ellipsis;
+            white-space: nowrap
+        }
+
+        .d2h-file-wrapper {
+            border: 1px solid #ddd;
+            border: 1px solid var(--d2h-border-color);
+            border-radius: 3px;
+            margin-bottom: 1em
+        }
+
+        .d2h-file-collapse {
+            -webkit-box-pack: end;
+            -ms-flex-pack: end;
+            cursor: pointer;
+            display: none;
+            font-size: 12px;
+            justify-content: flex-end;
+            -webkit-box-align: center;
+            -ms-flex-align: center;
+            align-items: center;
+            border: 1px solid #ddd;
+            border: 1px solid var(--d2h-border-color);
+            border-radius: 3px;
+            padding: 4px 8px
+        }
+
+        .d2h-file-collapse.d2h-selected {
+            background-color: #c8e1ff;
+            background-color: var(--d2h-selected-color)
+        }
+
+        .d2h-file-collapse-input {
+            margin: 0 4px 0 0
+        }
+
+        .d2h-diff-table {
+            border-collapse: collapse;
+            font-family: Menlo, Consolas, monospace;
+            font-size: 13px;
+            width: 100%
+        }
+
+        .d2h-files-diff {
+            display: -webkit-box;
+            display: -ms-flexbox;
+            display: flex;
+            width: 100%
+        }
+
+        .d2h-file-diff {
+            overflow-y: hidden
+        }
+
+        .d2h-file-diff.d2h-d-none, .d2h-files-diff.d2h-d-none {
+            display: none
+        }
+
+        .d2h-file-side-diff {
+            display: inline-block;
+            overflow-x: scroll;
+            overflow-y: hidden;
+            width: 50%
+        }
+
+        .d2h-code-line {
+            padding: 0 8em;
+            width: calc(100% - 16em)
+        }
+
+        .d2h-code-line, .d2h-code-side-line {
+            display: inline-block;
+            -webkit-user-select: none;
+            -moz-user-select: none;
+            -ms-user-select: none;
+            user-select: none;
+            white-space: nowrap
+        }
+
+        .d2h-code-side-line {
+            padding: 0 4.5em;
+            width: calc(100% - 9em)
+        }
+
+        .d2h-code-line-ctn {
+            background: none;
+            display: inline-block;
+            padding: 0;
+            word-wrap: normal;
+            -webkit-user-select: text;
+            -moz-user-select: text;
+            -ms-user-select: text;
+            user-select: text;
+            vertical-align: middle;
+            white-space: pre;
+            width: 100%
+        }
+
+        .d2h-code-line del, .d2h-code-side-line del {
+            background-color: #ffb6ba;
+            background-color: var(--d2h-del-highlight-bg-color)
+        }
+
+        .d2h-code-line del, .d2h-code-line ins, .d2h-code-side-line del, .d2h-code-side-line ins {
+            border-radius: .2em;
+            display: inline-block;
+            margin-top: -1px;
+            -webkit-text-decoration: none;
+            text-decoration: none
+        }
+
+        .d2h-code-line ins, .d2h-code-side-line ins {
+            background-color: #97f295;
+            background-color: var(--d2h-ins-highlight-bg-color);
+            text-align: left
+        }
+
+        .d2h-code-line-prefix {
+            background: none;
+            display: inline;
+            padding: 0;
+            word-wrap: normal;
+            white-space: pre
+        }
+
+        .line-num1 {
+            float: left
+        }
+
+        .line-num1, .line-num2 {
+            -webkit-box-sizing: border-box;
+            box-sizing: border-box;
+            overflow: hidden;
+            padding: 0 .5em;
+            text-overflow: ellipsis;
+            width: 3.5em
+        }
+
+        .line-num2 {
+            float: right
+        }
+
+        .d2h-code-linenumber {
+            background-color: #fff;
+            background-color: var(--d2h-bg-color);
+            border: solid #eee;
+            border: solid var(--d2h-line-border-color);
+            border-width: 0 1px;
+            -webkit-box-sizing: border-box;
+            box-sizing: border-box;
+            color: rgba(0, 0, 0, .3);
+            color: var(--d2h-dim-color);
+            cursor: pointer;
+            display: inline-block;
+            position: absolute;
+            text-align: right;
+            width: 7.5em
+        }
+
+        .d2h-code-linenumber:after {
+            content: "\200b"
+        }
+
+        .d2h-code-side-linenumber {
+            background-color: #fff;
+            background-color: var(--d2h-bg-color);
+            border: solid #eee;
+            border: solid var(--d2h-line-border-color);
+            border-width: 0 1px;
+            -webkit-box-sizing: border-box;
+            box-sizing: border-box;
+            color: rgba(0, 0, 0, .3);
+            color: var(--d2h-dim-color);
+            cursor: pointer;
+            display: inline-block;
+            overflow: hidden;
+            padding: 0 .5em;
+            position: absolute;
+            text-align: right;
+            text-overflow: ellipsis;
+            width: 4em
+        }
+
+        .d2h-code-side-linenumber:after {
+            content: "\200b"
+        }
+
+        .d2h-code-side-emptyplaceholder, .d2h-emptyplaceholder {
+            background-color: #f1f1f1;
+            background-color: var(--d2h-empty-placeholder-bg-color);
+            border-color: #e1e1e1;
+            border-color: var(--d2h-empty-placeholder-border-color)
+        }
+
+        .d2h-code-line-prefix, .d2h-code-linenumber, .d2h-code-side-linenumber, .d2h-emptyplaceholder {
+            -webkit-user-select: none;
+            -moz-user-select: none;
+            -ms-user-select: none;
+            user-select: none
+        }
+
+        .d2h-code-linenumber, .d2h-code-side-linenumber {
+            direction: rtl
+        }
+
+        .d2h-del {
+            background-color: #fee8e9;
+            background-color: var(--d2h-del-bg-color);
+            border-color: #e9aeae;
+            border-color: var(--d2h-del-border-color)
+        }
+
+        .d2h-ins {
+            background-color: #dfd;
+            background-color: var(--d2h-ins-bg-color);
+            border-color: #b4e2b4;
+            border-color: var(--d2h-ins-border-color)
+        }
+
+        .d2h-info {
+            background-color: #f8fafd;
+            background-color: var(--d2h-info-bg-color);
+            border-color: #d5e4f2;
+            border-color: var(--d2h-info-border-color);
+            color: rgba(0, 0, 0, .3);
+            color: var(--d2h-dim-color)
+        }
+
+        .d2h-file-diff .d2h-del.d2h-change {
+            background-color: #fdf2d0;
+            background-color: var(--d2h-change-del-color)
+        }
+
+        .d2h-file-diff .d2h-ins.d2h-change {
+            background-color: #ded;
+            background-color: var(--d2h-change-ins-color)
+        }
+
+        .d2h-file-list-wrapper {
+            margin-bottom: 10px
+        }
+
+        .d2h-file-list-wrapper a {
+            -webkit-text-decoration: none;
+            text-decoration: none
+        }
+
+        .d2h-file-list-wrapper a, .d2h-file-list-wrapper a:visited {
+            color: #3572b0;
+            color: var(--d2h-moved-label-color)
+        }
+
+        .d2h-file-list-header {
+            text-align: left
+        }
+
+        .d2h-file-list-title {
+            font-weight: 700
+        }
+
+        .d2h-file-list-line {
+            display: -webkit-box;
+            display: -ms-flexbox;
+            display: flex;
+            text-align: left
+        }
+
+        .d2h-file-list {
+            display: block;
+            list-style: none;
+            margin: 0;
+            padding: 0
+        }
+
+        .d2h-file-list > li {
+            border-bottom: 1px solid #ddd;
+            border-bottom: 1px solid var(--d2h-border-color);
+            margin: 0;
+            padding: 5px 10px
+        }
+
+        .d2h-file-list > li:last-child {
+            border-bottom: none
+        }
+
+        .d2h-file-switch {
+            cursor: pointer;
+            display: none;
+            font-size: 10px
+        }
+
+        .d2h-icon {
+            margin-right: 10px;
+            vertical-align: middle;
+            fill: currentColor
+        }
+
+        .d2h-deleted {
+            color: #c33;
+            color: var(--d2h-del-label-color)
+        }
+
+        .d2h-added {
+            color: #399839;
+            color: var(--d2h-ins-label-color)
+        }
+
+        .d2h-changed {
+            color: #d0b44c;
+            color: var(--d2h-change-label-color)
+        }
+
+        .d2h-moved {
+            color: #3572b0;
+            color: var(--d2h-moved-label-color)
+        }
+
+        .d2h-tag {
+            background-color: #fff;
+            background-color: var(--d2h-bg-color);
+            display: -webkit-box;
+            display: -ms-flexbox;
+            display: flex;
+            font-size: 10px;
+            margin-left: 5px;
+            padding: 0 2px
+        }
+
+        .d2h-deleted-tag {
+            border: 1px solid #c33;
+            border: 1px solid var(--d2h-del-label-color)
+        }
+
+        .d2h-added-tag {
+            border: 1px solid #399839;
+            border: 1px solid var(--d2h-ins-label-color)
+        }
+
+        .d2h-changed-tag {
+            border: 1px solid #d0b44c;
+            border: 1px solid var(--d2h-change-label-color)
+        }
+
+        .d2h-moved-tag {
+            border: 1px solid #3572b0;
+            border: 1px solid var(--d2h-moved-label-color)
+        }
+
+        .d2h-dark-color-scheme {
+            background-color: #0d1117;
+            background-color: var(--d2h-dark-bg-color);
+            color: #e6edf3;
+            color: var(--d2h-dark-color)
+        }
+
+        .d2h-dark-color-scheme .d2h-file-header {
+            background-color: #161b22;
+            background-color: var(--d2h-dark-file-header-bg-color);
+            border-bottom: #30363d;
+            border-bottom: var(--d2h-dark-file-header-border-color)
+        }
+
+        .d2h-dark-color-scheme .d2h-lines-added {
+            border: 1px solid rgba(46, 160, 67, .4);
+            border: 1px solid var(--d2h-dark-ins-border-color);
+            color: #3fb950;
+            color: var(--d2h-dark-ins-label-color)
+        }
+
+        .d2h-dark-color-scheme .d2h-lines-deleted {
+            border: 1px solid rgba(248, 81, 73, .4);
+            border: 1px solid var(--d2h-dark-del-border-color);
+            color: #f85149;
+            color: var(--d2h-dark-del-label-color)
+        }
+
+        .d2h-dark-color-scheme .d2h-code-line del, .d2h-dark-color-scheme .d2h-code-side-line del {
+            background-color: rgba(248, 81, 73, .4);
+            background-color: var(--d2h-dark-del-highlight-bg-color)
+        }
+
+        .d2h-dark-color-scheme .d2h-code-line ins, .d2h-dark-color-scheme .d2h-code-side-line ins {
+            background-color: rgba(46, 160, 67, .4);
+            background-color: var(--d2h-dark-ins-highlight-bg-color)
+        }
+
+        .d2h-dark-color-scheme .d2h-diff-tbody {
+            border-color: #30363d;
+            border-color: var(--d2h-dark-border-color)
+        }
+
+        .d2h-dark-color-scheme .d2h-code-side-linenumber {
+            background-color: #0d1117;
+            background-color: var(--d2h-dark-bg-color);
+            border-color: #21262d;
+            border-color: var(--d2h-dark-line-border-color);
+            color: #6e7681;
+            color: var(--d2h-dark-dim-color)
+        }
+
+        .d2h-dark-color-scheme .d2h-files-diff .d2h-code-side-emptyplaceholder, .d2h-dark-color-scheme .d2h-files-diff .d2h-emptyplaceholder {
+            background-color: hsla(215, 8%, 47%, .1);
+            background-color: var(--d2h-dark-empty-placeholder-bg-color);
+            border-color: #30363d;
+            border-color: var(--d2h-dark-empty-placeholder-border-color)
+        }
+
+        .d2h-dark-color-scheme .d2h-code-linenumber {
+            background-color: #0d1117;
+            background-color: var(--d2h-dark-bg-color);
+            border-color: #21262d;
+            border-color: var(--d2h-dark-line-border-color);
+            color: #6e7681;
+            color: var(--d2h-dark-dim-color)
+        }
+
+        .d2h-dark-color-scheme .d2h-del {
+            background-color: rgba(248, 81, 73, .1);
+            background-color: var(--d2h-dark-del-bg-color);
+            border-color: rgba(248, 81, 73, .4);
+            border-color: var(--d2h-dark-del-border-color)
+        }
+
+        .d2h-dark-color-scheme .d2h-ins {
+            background-color: rgba(46, 160, 67, .15);
+            background-color: var(--d2h-dark-ins-bg-color);
+            border-color: rgba(46, 160, 67, .4);
+            border-color: var(--d2h-dark-ins-border-color)
+        }
+
+        .d2h-dark-color-scheme .d2h-info {
+            background-color: rgba(56, 139, 253, .1);
+            background-color: var(--d2h-dark-info-bg-color);
+            border-color: rgba(56, 139, 253, .4);
+            border-color: var(--d2h-dark-info-border-color);
+            color: #6e7681;
+            color: var(--d2h-dark-dim-color)
+        }
+
+        .d2h-dark-color-scheme .d2h-file-diff .d2h-del.d2h-change {
+            background-color: rgba(210, 153, 34, .2);
+            background-color: var(--d2h-dark-change-del-color)
+        }
+
+        .d2h-dark-color-scheme .d2h-file-diff .d2h-ins.d2h-change {
+            background-color: rgba(46, 160, 67, .25);
+            background-color: var(--d2h-dark-change-ins-color)
+        }
+
+        .d2h-dark-color-scheme .d2h-file-wrapper {
+            border: 1px solid #30363d;
+            border: 1px solid var(--d2h-dark-border-color)
+        }
+
+        .d2h-dark-color-scheme .d2h-file-collapse {
+            border: 1px solid #0d1117;
+            border: 1px solid var(--d2h-dark-bg-color)
+        }
+
+        .d2h-dark-color-scheme .d2h-file-collapse.d2h-selected {
+            background-color: rgba(56, 139, 253, .1);
+            background-color: var(--d2h-dark-selected-color)
+        }
+
+        .d2h-dark-color-scheme .d2h-file-list-wrapper a, .d2h-dark-color-scheme .d2h-file-list-wrapper a:visited {
+            color: #3572b0;
+            color: var(--d2h-dark-moved-label-color)
+        }
+
+        .d2h-dark-color-scheme .d2h-file-list > li {
+            border-bottom: 1px solid #0d1117;
+            border-bottom: 1px solid var(--d2h-dark-bg-color)
+        }
+
+        .d2h-dark-color-scheme .d2h-deleted {
+            color: #f85149;
+            color: var(--d2h-dark-del-label-color)
+        }
+
+        .d2h-dark-color-scheme .d2h-added {
+            color: #3fb950;
+            color: var(--d2h-dark-ins-label-color)
+        }
+
+        .d2h-dark-color-scheme .d2h-changed {
+            color: #d29922;
+            color: var(--d2h-dark-change-label-color)
+        }
+
+        .d2h-dark-color-scheme .d2h-moved {
+            color: #3572b0;
+            color: var(--d2h-dark-moved-label-color)
+        }
+
+        .d2h-dark-color-scheme .d2h-tag {
+            background-color: #0d1117;
+            background-color: var(--d2h-dark-bg-color)
+        }
+
+        .d2h-dark-color-scheme .d2h-deleted-tag {
+            border: 1px solid #f85149;
+            border: 1px solid var(--d2h-dark-del-label-color)
+        }
+
+        .d2h-dark-color-scheme .d2h-added-tag {
+            border: 1px solid #3fb950;
+            border: 1px solid var(--d2h-dark-ins-label-color)
+        }
+
+        .d2h-dark-color-scheme .d2h-changed-tag {
+            border: 1px solid #d29922;
+            border: 1px solid var(--d2h-dark-change-label-color)
+        }
+
+        .d2h-dark-color-scheme .d2h-moved-tag {
+            border: 1px solid #3572b0;
+            border: 1px solid var(--d2h-dark-moved-label-color)
+        }
+
+        @media (prefers-color-scheme: dark) {
+            .d2h-auto-color-scheme {
+                background-color: #0d1117;
+                background-color: var(--d2h-dark-bg-color);
+                color: #e6edf3;
+                color: var(--d2h-dark-color)
+            }
+
+            .d2h-auto-color-scheme .d2h-file-header {
+                background-color: #161b22;
+                background-color: var(--d2h-dark-file-header-bg-color);
+                border-bottom: #30363d;
+                border-bottom: var(--d2h-dark-file-header-border-color)
+            }
+
+            .d2h-auto-color-scheme .d2h-lines-added {
+                border: 1px solid rgba(46, 160, 67, .4);
+                border: 1px solid var(--d2h-dark-ins-border-color);
+                color: #3fb950;
+                color: var(--d2h-dark-ins-label-color)
+            }
+
+            .d2h-auto-color-scheme .d2h-lines-deleted {
+                border: 1px solid rgba(248, 81, 73, .4);
+                border: 1px solid var(--d2h-dark-del-border-color);
+                color: #f85149;
+                color: var(--d2h-dark-del-label-color)
+            }
+
+            .d2h-auto-color-scheme .d2h-code-line del, .d2h-auto-color-scheme .d2h-code-side-line del {
+                background-color: rgba(248, 81, 73, .4);
+                background-color: var(--d2h-dark-del-highlight-bg-color)
+            }
+
+            .d2h-auto-color-scheme .d2h-code-line ins, .d2h-auto-color-scheme .d2h-code-side-line ins {
+                background-color: rgba(46, 160, 67, .4);
+                background-color: var(--d2h-dark-ins-highlight-bg-color)
+            }
+
+            .d2h-auto-color-scheme .d2h-diff-tbody {
+                border-color: #30363d;
+                border-color: var(--d2h-dark-border-color)
+            }
+
+            .d2h-auto-color-scheme .d2h-code-side-linenumber {
+                background-color: #0d1117;
+                background-color: var(--d2h-dark-bg-color);
+                border-color: #21262d;
+                border-color: var(--d2h-dark-line-border-color);
+                color: #6e7681;
+                color: var(--d2h-dark-dim-color)
+            }
+
+            .d2h-auto-color-scheme .d2h-files-diff .d2h-code-side-emptyplaceholder, .d2h-auto-color-scheme .d2h-files-diff .d2h-emptyplaceholder {
+                background-color: hsla(215, 8%, 47%, .1);
+                background-color: var(--d2h-dark-empty-placeholder-bg-color);
+                border-color: #30363d;
+                border-color: var(--d2h-dark-empty-placeholder-border-color)
+            }
+
+            .d2h-auto-color-scheme .d2h-code-linenumber {
+                background-color: #0d1117;
+                background-color: var(--d2h-dark-bg-color);
+                border-color: #21262d;
+                border-color: var(--d2h-dark-line-border-color);
+                color: #6e7681;
+                color: var(--d2h-dark-dim-color)
+            }
+
+            .d2h-auto-color-scheme .d2h-del {
+                background-color: rgba(248, 81, 73, .1);
+                background-color: var(--d2h-dark-del-bg-color);
+                border-color: rgba(248, 81, 73, .4);
+                border-color: var(--d2h-dark-del-border-color)
+            }
+
+            .d2h-auto-color-scheme .d2h-ins {
+                background-color: rgba(46, 160, 67, .15);
+                background-color: var(--d2h-dark-ins-bg-color);
+                border-color: rgba(46, 160, 67, .4);
+                border-color: var(--d2h-dark-ins-border-color)
+            }
+
+            .d2h-auto-color-scheme .d2h-info {
+                background-color: rgba(56, 139, 253, .1);
+                background-color: var(--d2h-dark-info-bg-color);
+                border-color: rgba(56, 139, 253, .4);
+                border-color: var(--d2h-dark-info-border-color);
+                color: #6e7681;
+                color: var(--d2h-dark-dim-color)
+            }
+
+            .d2h-auto-color-scheme .d2h-file-diff .d2h-del.d2h-change {
+                background-color: rgba(210, 153, 34, .2);
+                background-color: var(--d2h-dark-change-del-color)
+            }
+
+            .d2h-auto-color-scheme .d2h-file-diff .d2h-ins.d2h-change {
+                background-color: rgba(46, 160, 67, .25);
+                background-color: var(--d2h-dark-change-ins-color)
+            }
+
+            .d2h-auto-color-scheme .d2h-file-wrapper {
+                border: 1px solid #30363d;
+                border: 1px solid var(--d2h-dark-border-color)
+            }
+
+            .d2h-auto-color-scheme .d2h-file-collapse {
+                border: 1px solid #0d1117;
+                border: 1px solid var(--d2h-dark-bg-color)
+            }
+
+            .d2h-auto-color-scheme .d2h-file-collapse.d2h-selected {
+                background-color: rgba(56, 139, 253, .1);
+                background-color: var(--d2h-dark-selected-color)
+            }
+
+            .d2h-auto-color-scheme .d2h-file-list-wrapper a, .d2h-auto-color-scheme .d2h-file-list-wrapper a:visited {
+                color: #3572b0;
+                color: var(--d2h-dark-moved-label-color)
+            }
+
+            .d2h-auto-color-scheme .d2h-file-list > li {
+                border-bottom: 1px solid #0d1117;
+                border-bottom: 1px solid var(--d2h-dark-bg-color)
+            }
+
+            .d2h-dark-color-scheme .d2h-deleted {
+                color: #f85149;
+                color: var(--d2h-dark-del-label-color)
+            }
+
+            .d2h-auto-color-scheme .d2h-added {
+                color: #3fb950;
+                color: var(--d2h-dark-ins-label-color)
+            }
+
+            .d2h-auto-color-scheme .d2h-changed {
+                color: #d29922;
+                color: var(--d2h-dark-change-label-color)
+            }
+
+            .d2h-auto-color-scheme .d2h-moved {
+                color: #3572b0;
+                color: var(--d2h-dark-moved-label-color)
+            }
+
+            .d2h-auto-color-scheme .d2h-tag {
+                background-color: #0d1117;
+                background-color: var(--d2h-dark-bg-color)
+            }
+
+            .d2h-auto-color-scheme .d2h-deleted-tag {
+                border: 1px solid #f85149;
+                border: 1px solid var(--d2h-dark-del-label-color)
+            }
+
+            .d2h-auto-color-scheme .d2h-added-tag {
+                border: 1px solid #3fb950;
+                border: 1px solid var(--d2h-dark-ins-label-color)
+            }
+
+            .d2h-auto-color-scheme .d2h-changed-tag {
+                border: 1px solid #d29922;
+                border: 1px solid var(--d2h-dark-change-label-color)
+            }
+
+            .d2h-auto-color-scheme .d2h-moved-tag {
+                border: 1px solid #3572b0;
+                border: 1px solid var(--d2h-dark-moved-label-color)
+            }
+        }
+    }
+}

survey_guide.md

@@ -0,0 +1,19 @@
+**Thank you for agreeing to take part!**
+
+In this study, we study how developers interact with 
+commit message text generated by AI Assistant. Imagine 
+that you are in one of the JetBrains IDEs and writing a 
+commit message (the changes to the code are shown on the 
+left). You have generated a message using AI Assistant. 
+You can see this generated message on the top right.
+However, the message will not necessarily be perfect! 
+
+Now, _you need to edit the commit message_ so that it is good enough 
+to publish to VCS. Please do this directly in the text 
+box. When you're done, press the "Submit" button: you
+will proceed to the next commit and the next generated
+message. Please, repeat the process for as many commits
+as you can.
+
+During the study, we collect all keystrokes made _in a 
+text window_, as well as the resulting message,