.. _management: .. title:: Coquery Documentation: Data management Data management =============== Once the query from the data base is completed, the results can be managed in different ways. You can: * Change the order of the columns in the results table * Specify the context that are displayed for each matched token * Specify stopwords that are filtered out from the results * Split the results into data groups, apply functions to each group, and merge the groups again * Transform the results table into another type of table * Apply summary functions and filters that remove matches from the results You can also apply user functions to each column in the results table. The data management toolbox in the upper right corner of the main screen provides access to most of these features. .. figure:: ../../_static/interface/tutorial_aggregation_none.png :scale: 100 % Feature order ++++++++++++ TO BE ADDED Context +++++++ The **Context** toolbox is used to specify how much context to the left and to the right of the matched token should be displayed. The two controls *Left:* and *Right:* indicate the number of words in the left and in the right context. The value of these settings is also used by the *Collocation list* transformation described below. This toolbox also controls how the context is shown: * *Columns*: Each word in the context is shown in a separate column in the results table. This view has the advantage that you can sort the tokens in the query by words in the context. * *KWIC*: All words from the left context are shown in one column in the results table, and all words from the right coontext are shown in another. This view resembles the display of traditional concordancers. * *Single string*: The whole context as well as the matched token are shown in a single column. The words that are a part of the match are shown in capital letters so that they are easier to spot in the context. If the current corpus contains information about sentence boundaries (e.g. the BNC), the context can be restricted to sentences by checking the box *Restrict to sentences*. If the contexts are restricted in this way, the number of words that are shown can be smaller than the number of words selected by the *Left:* and *Right:* controls if the sentence is shorter than these values. Retrieving contexts can take some time. If you do not want to include the contexts in your results table, choose *No context*. Stopwords +++++++++ TO BE ADDED Data groups +++++++++++ The matched tokens can be temporarily split into data groups so that all matches in each group have the same value in one or more of their columns. For example, data groups may be specified by using the part-of-speech column. In that case, all tokens in the groups share the same part-of-speech tag. If the file name column is also added, each part-of-speech tag group is further subdivided into matches from the same files. Only those columns can be used for data groups that are included in the data table. This includes functions and columns that are produced by data transformations. For each data group, a set of group functions can be specified. These functions consider only the matches that are contained in each group, and ignore the matches in all other groups. By using group functions, it is for instance easy to obtain the type-token ratio for different files in a corpus. In order to do so, use the query string `*` to match all tokens in the corpus (which may take a while with larger corpora). Select `Word` and `Filename` (if available) as output features, and press *New query*. Now, specify a new data group by pressing the *New* button in the *Data groups* toolbox. The **Data group dialog** will appear. In this dialog, select the column `Filename` as the only grouping column. Scroll down in the list of group functions until you find the *Type-token ratio* function. Press the *Change columns* button, and select the `Word` column. If the data management is applied, there will be a new column `TTR(Word by Filename)` which shows the type-token ratio in each file. Transformations +++++++++++++++ Only distinct rows ------------------ This transformation discards all doublets, i.e. each row that has the same values in its columns as a previous row. Frequency list -------------- The frequency list shows the number of occurrences of the values in the results table. If you want to exclude one of the columns from the frequency count, you can hide that column by right-clicking column header, and selecting "Hide column" from the context menu. Contingency table ----------------- A contingcy tables is a restructured feature list. The last entry in the feature order toolbox is the variable that defines the shape of the contingency table. Each value of that variable forms a separate column in the table. The contingency table also shows row and column totals. Currently, the behavior of numeric values is undefined. G-test matrix ------------- The G-test matrix takes the results table and applies pairwise log-likelihood tests to determine which results occur with statistically different frequencies in the current corpus. The figure shown in the matrix is the *G* statistic. Colors in the matrix indicate a statistically significant value. A red cell indicates that the result in the row is significantly more frequent than the result in the column. Depending on the number of value combinations, the test matrix can become relatively large, and therefore, many multiple comparisons can be performed. To correct for an inflation of significant test results, a corrected alpha value is determined using the False Discovery Rate method (Benjamini & Hochberg 1995, described in Narum 2006). If there are corpus features in the results table, these features are used to construct subcorpora. In this way, it is easy to compare for instance the number of tokens between female and male speakers, or between different files. If you click on a cell in the G-test matrix, the *Independence test viewer* opens. This dialog shows a contingency table, and calculates the corresponding log-likelihood test and Chi-square test. It also estimates the effect size by displaying the odds ratio and Cramer's phi. The dialog offers buttons to copy the content of the dialog to the clipboard, either as HTML, LaTeX, or als plain text. **Warning:** The G-test matrix is still a somewhat experimental feature. If you wish to report the results of the involved statistical tests in a scientific publication, you are strongly advised to check their validity first. Collocation list ---------------- 'Collocations' determines words that frequently co-occur with the words from the query strings. The span of words to the left and to the right that are considered for the collocation list is defined in the *Context* toolbox.