asciimoo

joined 2 months ago
[–] asciimoo@lemmy.ml 2 points 5 hours ago (1 children)

Currently it is not planned. Hister guarantees that non of your data/query/metadata leaves the service if you use it. As I see, this is a more valuable and unique feature than having an integrated metasearch. There are already great metasearch solutions and Hister provides an easy fallback to search providers, so in my opinion this direction would be more of a sacrifice than an improvement.

[–] asciimoo@lemmy.ml 1 points 1 day ago (1 children)

The results are standard search results, if you click on the title, it opens the original URL, just like in any other regular search engine. You can even close the preview pane if you are not interested in it.

[–] asciimoo@lemmy.ml 6 points 1 day ago (1 children)

However, dismissing it as “a multiple screens long AI prompt” does not only not answer the question, it comes off as abrasive.

I think it is more abrasive to copy/paste a poorly formatted LLM output instead of taking the time and summarizing it to a few sentences just as you did in your previous post.

[–] asciimoo@lemmy.ml 5 points 1 day ago* (last edited 1 day ago) (3 children)

Is there a TTL / max database size per user setting?

As I wrote, you can simply automate deletion by document age. Schedule a delete event on each day with the desired retention time defined as a filter expression. Database size limit isn't available yet.

Additionally, is the other parenthetical information materially correct? If not, which points [1 thru to 7] are wrong?

No, sorry, I don't have time to correct a copy of a multiple screens long AI prompt.

I would like to further recommend Hister but your documentation is somewhat confusing at first blush.

Which parts are confusing?

[–] asciimoo@lemmy.ml 11 points 1 day ago (6 children)

The summary has numerous inaccuracies. Most importantly: it is pretty easy to delete content by topic or age. The hister delete command can accept a search query to remove only matched documents. The same is true on the web UI "actions -> remove all matching documents". You can quickly filter by age, simply query updated:>365d. Combine it with URLs, labels, domains or phrases.

[–] asciimoo@lemmy.ml 1 points 1 day ago

it’s a habit changer.

It took some time to get used to it, but I find it more efficient, to spawn a Hister with a single hotkey and use the URL bar with a traditional search engine when I know that I'm looking for something new. But, of course this is subjective, everbody has their preferred workflows, I shared mine as an inspiration.

Btw, you can start your query with !!, in this case Hister automatically redirects you to your external search engine with the rest of the search query.

[–] asciimoo@lemmy.ml 2 points 1 day ago

I've started creating smaller, pre-fetched, thematic datasets: https://hister.org/datasets . It is still very rudimentary, but the concept is the same. Both datasets and improvement ideas are appreciated.

[–] asciimoo@lemmy.ml 1 points 1 day ago (2 children)

It isn’t possible to setup hister as my default search engine in browser with automated fallback to normal Meta-search with whatever engine?

It can be added as a default search engine and it provides a link and a hotkey to open the search query with an external search engine, however I have a different workflow: https://hister.org/posts/how-i-use-hister

And I use login credentials maybe a faster way like passkey possible in the future?

That's a good idea, thanks for the suggestion.

[–] asciimoo@lemmy.ml 2 points 1 day ago

Though maybe the resource folder can be ignored and the html file can already be imported?

It depends. Hister always requires a unique URL for each document. SingleFile snapshots include the original URL of the document as a meta HTML element. I'm not sure if the built-in page saver provides URL information.

[–] asciimoo@lemmy.ml 7 points 1 day ago

I'd appreciate it, thanks. No special requirements. Providing sensible defaults and explaining usage/potential customization options would be great.

[–] asciimoo@lemmy.ml 3 points 1 day ago

The storage requirement is around 100KB/page on average.

Memory usage can exceed 1GB momentarily for searches when using language detection and multi-language indexes (it is the default config). Without language detection Hister has a much smaller memory footprint (~30MB default with ~100-150MB peaks).

[–] asciimoo@lemmy.ml 16 points 1 day ago

Exactly, the echo chamber phenomenon is mostly problematic for "discovery type" searches while Hister is mainly for "recall type" search.

Implementing federated search/index sharing could be a partial solution to this issue in the long run.

 

Hi everyone,

I am the original author of Searx. I started Hister with a similar motivation: reducing our dependence on external search engines while keeping searches and personal data under our control.

Searx is a metasearch engine that forwards queries to other search providers. Hister takes a different approach. It builds a private full text index from content you choose, then searches that index entirely on your own infrastructure.

Hister can automatically index pages through its Firefox and Chrome extensions. It can also watch local directories, import browser history and bookmarks, index individual URLs, and crawl complete documentation sites.

The feature I find most useful is offline previews. Hister stores the readable content and HTML of indexed pages locally. You can open a result in a clean and sanitized preview beside the search results without visiting the original website again.

Some other features:

  1. Full text search across web pages, PDFs, docx files, Markdown, OrgMode and text files
  2. Phrase searches, field filters, date filters, wildcards, negation, aliases, labels, facets, and result priorities
  3. Optional semantic search using an embeddings endpoint you configure
  4. Persistent website crawls
  5. Imports from browser history, Linkwarden, Karakeep, Shaarli, Wallabag, and Linkding
  6. Web, terminal, command line, HTTP API, and MCP interfaces
  7. SQLite and PostgreSQL support, plus optional multiple user hosting

Hister cannot replace a global search engine (yet) for subjects you have never encountered because it only searches what you have indexed. My workflow is to search Hister first, then use its shortcut to fall back to traditional search when I need broader web results.

The project is free software under the AGPLv3+ license. It can be installed as a standalone binary or with Docker.

Project: https://github.com/asciimoo/hister

Website and documentation: https://hister.org/

Small read-only demo: https://demo.hister.org/

I'd appreciate feedback, questions, and suggestions as well as joining our growing community.

AI disclosure: AI assisted contributions are not strictly prohibited, but all contributions should be made by humans. More details: https://github.com/asciimoo/hister/blob/master/CONTRIBUTING.md#ai-policy

 

I'm working on a self-hosted search service called Hister with the goal to reduce my dependence on online search engines.

Hister is a full text indexer for websites which saves all the visited pages rendered by your browser. It provides a flexible web (and terminal) search interface & query language to explore previously visited content with ease or quickly fall back to traditional search engines.

I've been using it for a few months and as my local index is growing I can avoid opening google/duckduckgo/kagi more and more frequently.

The project is still heavily under development with a growing community, but the current version is in a fairly usable state in my opinion, so I wanted to share it here - perhaps some of you find it useful as well. (Or at least have some constructive criticism =])

The code is AGPLv3 licensed, available at https://github.com/asciimoo/hister website: https://hister.org/ read-only demo: https://demo.hister.org/

About me: I develop privacy protecting and data liberating free software since 2008. I'm the author of Searx, Colly (https://github.com/gocolly/colly) and many more smaller free software/self-hosted projects (https://github.com/asciimoo).

view more: next ›