Tag Hierarchy for vulpea

[Cletip]

Hello,

My question is in the title: would it be possible to implement the functionality of having a tag hierarchy in vulpea?

Concrete example:

If I have the following configuration (https://orgmode.org/manual/Tag-Hierarchy.html)

(setq org-tag-alist '((:startgrouptag)
                      ("GTD")
                      (:grouptags)
                      ("Control")
                      ("Persp")
                      (:endgrouptag)
                      (:startgrouptag)
                      (:Control")
                      (:grouptags)
                      ("Context")
                      (:Task")
                      (:endgrouptag))

That can conceptually be seen as a hierarchy of tags like this:

GTD
  Persp
    Vision
    Goal
    AOF
    Project
  Control
    Context
    Task

If one of the notes has for example the tag "Task" (in filetag), would it be possible that, when using the command "vulpea-db-query-by-tags-every", this note would be found if I ask for the tag "Control" or even "GTD"?

To conclude, I may not realize the difficulty of this feature :

Is it possible ?
Could there be alternatives ?
Is it complicated to implement ?

This feature would just be execeptional for me,

Thanks in advance for your futur answer.

[d12frosted]

Hey,

I am in a travel mode, so can't answer in full details. But from the top of my mind, it should work because vulpea doesn't have it's own notion of tags. Everything is done using Org mode APIs. So I'd say - if vulpea doesn't respect tags configuration, then it's a bug.

What I mean is that if org-get-tags returns tags you are looking for, it would work. If org-get-tags doesn't return "Control" or "GTD" then 🤷 I would rather look on how to configure org-mode.

Meanwhile I will read that article about hierarchy, cause I know little about that.

[Cletip]

Thank you for your answer.

Okay, I think I just realized that there are two major problems:

1. the first one is about headings (not file-level notes). The "org-get-tags" command does not return "super tags". To use my previous example, if I put "Control" to a heading, org-get-tag will only return "Control", not "GTD Control". Also, strangely enough, the sparce-tree works perfectly for the tag hierarchy."

2. Now, the "file-note". These are, I think, not taken into account in the native org-mode (tags are only for headings, not files I think. Putting "#+filetags:" at the beginning of the file means (from the documentation of org-mode) : 
   "You can also set tags that all entries in a file should inherit just as if these tags were defined in a hypothetical level zero that surrounds the entire file."
   But this does not mean for org-mode that the file has this tag. So you would have to store the tags for each file/note. I looked at your "vulpea-db--schemata", and you already store the tags! So a hierarchy could be implemented with a logical sequence of or?
   Example: wanting the tag "Control" would be like doing the command
   (vulpea-db-query-by-tags-some '("Control" "Task" "Context")).

Then I see two solutions: either the tags are returned by a function, a little like this :
(vulpea-db-query-by-tags-some (function-to-return-a-list-of-tags "Control")) which would take into account the "org-tag-alist" variable (or another variable that describe a hierarchy). But I don't think this is the right solution, there would be too many things to change (I think, I don't realize really the consequences)

Or you have to implement it differently with vulpea. The idea would be maybe to rewrite the function that allows to have the tags of a note in vulpea, which would take into account the "org-tag-alist" variable directly.

There are surely other solutions that I haven't thought of.

Finally, why not create another variable to store the tag hierarchy (exclusive to vulpea, why not just a list of list of list etc). Its initial value could be parsed with the basic "org-tag-alist" variable, but the user could change it to give another hierarchy than the one by org-mode.

Also, thank you very much for this wonderful package and for your involvement, whether in your blogs, message or simply your different projects (I see the new repo "publicatorg").

I would like to help more with the code, unfortunately I don't have the level to do it yet.

Thanks in advance for your future answer

[d12frosted]

@Cletip sorry for the delay. I was playing around wit this idea of using tag groups. And in the process I realised that this issomething I would use myself in variety of contexts. For example, for my vino or litnotes plugins.

The major problem I see right now is that Org mode does not provide a way¹ to get a list of tags including their groups. E.g. in your example, to get ("GTD" "Control" "Task") for entry tagged as "Task".

The way I see it - org-get-tags should/could return groups in the same way as it returns inherited tags. But I found only helpers to create matchers for tag together with its groups - org-tags-expand (used by agenda search). But this is not that helpful.

In addition, the solution should be 'portable' to support file level notes via filetags.

I might work on such a function. But in really, I am not sure if that is "valid" flow 🤷

Now regarding your message. Thanks for taking time and checking various cases and sharing your thoughts.

the first one is about headings (not file-level notes). The "org-get-tags" command does not return "super tags". To use my previous example, if I put "Control" to a heading, org-get-tag will only return "Control", not "GTD Control". Also, strangely enough, the sparce-tree works perfectly for the tag hierarchy."

Indeed, this is what I found myself. That function supports inherited tags, but not groups.

Now, the "file-note". These are, I think, not taken into account in the native org-mode (tags are only for headings, not files I think. Putting "#+filetags:" at the beginning of the file means (from the documentation of org-mode) : 

Files are treated specially in Org Roam and (inherently) Vulpea. There is a code path that reads filetags value.

So a hierarchy could be implemented with a logical sequence of or?

Not sure I understand the solution. If the tag is in the DB, querying is not a problem. The problem is that there is no simple way to get the list of tags together with their respective groups (recursively). Once there is a way to do it - querying is not a problem.

(vulpea-db-query-by-tags-some (function-to-return-a-list-of-tags "Control")) which would take into account the "org-tag-alist" variable (or another variable that describe a hierarchy). But I don't think this is the right solution, there would be too many things to change (I think, I don't realize really the consequences)

Keep in mind that tags configurations are buffer-local, e.g. buffer content may change semantics of any tag. For performance and (probably) consistency it's better to store all information on write and avoid reading files when querying.

Finally, why not create another variable to store the tag hierarchy (exclusive to vulpea, why not just a list of list of list etc). Its initial value could be parsed with the basic "org-tag-alist" variable, but the user could change it to give another hierarchy than the one by org-mode.

I would try to stick to Org mode as much as possible :) That way it integrates with other parts of Org Mode.

---

So to summarise, to me it seems like all we need is a function that can expand tags to their groups. Using it we can implement a variant of org-get-tags that includes groups. Then we can advice/replace org-get-tags with this new variant and voila! Org Roam and Vulpea now support groups :)

Let me know if something is not clear in my answer.

---

Also, thank you very much for this wonderful package and for your involvement, whether in your blogs, message or simply your different projects (I see the new repo "publicatorg").

❤️

Footnotes

1. or at least I could not find one ↩

[Cletip]

Hi! Sorry for the little delay too

Yes I understood your answer, and I thank you for the precicion of this one.

I think indeed that this is probably the best solution.

I'll send a mail to the org-mode mailing list, someone surely did it already.

PS : I think that this functionnality : "In addition, the solution should be 'portable' to support file level notes via filetags." will have to be implemented by ourselves.

another PS : Have you seen the new package called "denote"? What do you think about it ? Can we discuss it somewhere else than this issue ?

[d12frosted]

@Cletip Cool. Let's see what we can get.

Have you seen the new package called "denote"? What do you think about it ? Can we discuss it somewhere else than this issue ?

Sure. Feel free to start a new public discussion or to send me an email.

[Cletip]

Hello,

I have some news!

You can have the "tags that are part of the hierarchy" with this function: "org-tags-expand".

But this function doesn't give the list of tags of a heading (it just takes one tag as argument)

I made a small function to illustrate the fact to get all the tags, using the "org-tag-alist" variable of org-mode.

You can try this in an org file:

#+TAGS: [ GTD : salut Persp Control ]
#+TAGS: [ Control : Context Task ]
#+TAGS: [ Persp : Vision Goal AOF Project ]

The function :

(defun org-get-tags-with-hierarchie()
      "Return the list of tag WITH the sub-tags if they exist"
      (interactive)
      (let ((tags-heading (org-get-tags))
            (tags-result '()))
        (dolist (tag tags-heading)
          (dolist (tag-to-add (org-tags-expand tag t))
            (push tag-to-add tags-result)
            )
          )
        (delete-dups tags-result)
        )
      )

Just execute the function when your cursor is on a heading with the tag "Persp" for example.

This allows, for a certain heading, to recover its tags. we can just modify the "tags-heading" variable to retrieve the tags of a note! I think

PS : I asked to know if a function of this style already existed in org-mode, I wait for the answer but it seems that not

[d12frosted]

@Cletip Yup, org-tags-expand is the function I've mentioned (it's used by agenda routine), but I couldn't quickly figure out how to use it for hierarchies 🤔 And seems like you figured that out.

Now with this function you can basically advice org-get-tags to return expanded list of tags and that way your Org Roam database will be filled with extra tags from hierarchy. Let me know if you need help with this.

P.S. I am not really sure that org-get-tags-with-hierarchie works semantically correct in the case of Persp. If I tag something as Persp, I don't expect it to have all of it's children as tags. I mean, if something is a Vision, then it's a Persp. But if it's a Persp, it's not necessarily a Vision. 🤷

[Cletip]

Sorry I'm late with the answer, really sorry.

Yup, org-tags-expand is the function I've mentioned (it's used by
agenda routine), but I couldn't quickly figure out how to use it for hierarchies thinking And seems like you figured that out.

Yes sorry, I forgot you had mentioned that.

So the principle is the following : get, according to the hierarchy given by the "org-tag-alist" variable, the children of a tag X, and give this tag list to the "vulpea-db-query-by-tags-some" function.

This allows to change only the search functions, and not to modify directly the tags of a note

  (defun cp/org-get-tags-with-children(tags)
    "Take a list of tag, and return this list of tag WITH the children (define in org-tag-alist) of each tag in entry"
    (interactive)
    (let (tags-result)
      (dolist (tag tags)
        (dolist (tag-to-add (org-tags-expand tag t))
          (push tag-to-add tags-result)))
      (delete-dups tags-result)
      )
    )

  (defun cp/vulpea-select-from-tags-with-children (tags)
    "Takes a list of tags, and allows the user to choose a note that has one of these tags OR has a child tag from the list given in parameter"
    (let ((links (vulpea-db-query-by-tags-some (cp/org-get-tags-with-children tags))))
      (unless links
        (user-error "There are note with the current tag (or children)"))
      (vulpea-find
       :candidates-fn (lambda (_) links)
       :require-match t))
    )
;;try
  (cp/vulpea-select-from-tags-with-children '("Persp"))

Here's what it does:

• define a hierarchy with the "org-tag-alist" variable
• put tags to the notes (so far, nothing unusual)
• search, thanks to the function "cp/vulpea-select-from-tags-with-children", the list of notes that respects the hierarchy. For example, using the function like this: 
  (cp/vulpea-select-from-tags-with-children '("Persp"))
  I would have all the notes with one of the following tags: "Persp Vision Goal AOF Project", as defined by the hierarchy defined at the beginning.
  On the other hand, if I use the tag "Vision", I would have only the notes with the tag "Vision".

With this function, you can advise org-get-tags to return an extended list of tags and thus your Org Roam database will be filled with additional tags from the hierarchy. Let me know if you need help with this.

I don't really understand this: do you mean something like this
(advice-add 'org-get-tags :after #'cp/org-get-tags-with-children)
? I don't really know how to "add a result", I know how to run before and after functions, but adding to a result is a problem for me. Not only because I don't know how to do it, but maybe just because it's dangerous to add an advice to the "org-get-tags" function

Also, I don't think we need to add an advice: why not just add a boolean in the search functions that would respect the hierarchy? What do you think about it?

It would look like this:

(vulpea-db-query-by-tags-some '("Persp")) ;; return only the note with the tag "Persp"
(vulpea-db-query-by-tags-some '("Persp") :hierarchie t) ;; return the note with tag Persp OR Goal OR AOF OR Project

I'm not really sure that org-get-tags-with-hierarchy works semantically correct in the case of Persp. If I tag something as Persp, I don't expect all its children to be tags. I mean, if something is a Vision, then it's a Persp. But if it's a Persp, it's not necessarily a Vision.

Okay, I think there's a definitional problem in the thing:

A hierarchy allows, for me, to do this: 

• create "super-tags" that group
• make searches that respect the hierarchy. If we take the example from the beginning, searching for the tag "control" will be like searching for the headings/notes that have the tag control OR context OR Task. On the other hand, if we just search for "Task", then only the notes with the "Task" tag will be searched (except if "Task" is a super-tag)

There can be several places, take here the example X

GTD
  Persp
    X
    Goal
    AOF
    Project
  Control
    Context
    Task
    X

Here it is stored in two places. So no matter if I call Control or Persp, it will always give all the notes with X, which is convenient. This allows you to store these tags in several places at once

Finally, I wanted to thank you again for taking the time to discuss these things with a complete stranger: me x)

[Cletip]

Hello

I'll be back to "help" you and to add some code yeah. It's not much after all, but I hope it works.

I finished a simple but efficient code that allows to simulate a hierarchy when searching.

This is done in two functions:

1. The first one allows to return, for a list of tags, all these sub tags (and themselves too)
2. The second one uses "vulpea-db-query-by-tags-some", to search for all the corresponding tags.

Both functions:

(defun cp/org-get-tags-with-children(tags)
    "Take a list of tag, and return this list of tag WITH the sub-tags (define in org-tag-alist) of each tag in entry"
    (interactive)
    (let (tags-result)
      (dolist (tag tags)
        (dolist (tag-to-add (org-tags-expand tag t))
          (push tag-to-add tags-result)))
      (delete-dups tags-result)
      )
    )

  (defun cp/vulpea-select-from-tags-with-children (tags)
    "Takes a list of tags, and allows the user to choose a note that has one of these tags OR has a child tag from the list given in parameter"
    (let ((links (vulpea-db-query-by-tags-some (cp/org-get-tags-with-children tags))))
      (unless links
        (user-error "There are note with the current tag (or children)"))
      (vulpea-find
       :candidates-fn (lambda (_) links)
       :require-match t))
    )

You can try, for example, by setting your note with the tag "Vision", and search with "Persp" (if you take the previous exemple with GTD, Persp etc), like this:
(cp/vulpea-select-from-tags-with-children '("Persp"))

PS : I don't know how you want to "implement" it in vulpea. I think it might be useful to have it in vulpea though.