Data Programming for Learning Discourse Structure

Work in progress_ ACL 2019

Code

Candidate generation:

The candidates are all pairs of segments or dialogue units (DUs) in a dialogue between which a attachement (of some relation type) is possible. Before applying the LFs to the STAC conversation data, we transformed the the dialogues into sets of candidates with the process described here.

Individual LFs:

• Acknowledgement LF
• Comment LF
• Conditional LF
• Continuation LF
• Contrast LF
• Elaboration LF
• Question-answer pair (QAP) LF
• Result LF
• Sequence LF

*Helper functions (called by LF functions)

LF performances on the STAC development set

In order to write a set of LFs/rules which adequately covered the data, we had to find a way to reasonably divide and conquer the myriad characteristics of the relations. We started by focusing on relation type: for each of the nine most frequent relation types, we wrote a separate rule for each of the sets of endpoint types most prevalent for that relation. Result (RES) is the only relation type which was found between all four endpoint permutations: LL (linguistic source-linguistic target), LNL (linguistic source, non-linguistic target), etc..

We used the development set, which consists of 3 games (10% of the STAC corpus) to write and revise the rules. The table below shows the performance of each rule on the percentage of the development set it covers, i.e. the percentage of the dev set containing relations with a particular combination of endpoint relations. For example, the LF for Question-answer-pair between Linguistic DUs only (QAP LL) has a coverage of (was applied to) 32% of the development set and an accuracy of 89%.