In the last few weeks, I frequently see some empty comments. It's just the username and no text beneath.

Is there a deeper reason behind this? Do people nowadays strip away the text instead of deleting a comment? Or did some script surface that 'makes the internet forget'? First I thought people did this before deleting a comment and the deletion just didn't get federated. But I scrolled through some older posts and they also still have comments like that, so that can't be it. Right?

Can anyone educate me?

Does it work well? Which one to choose? The official Matrix site shows 3 that seem maintained:

• matrix-appservice-discord
• mautrix-discord
• out-of-your-element

Does anyone have some insight? I don't want to try all of them.

Edit: I don't need anything super fancy like double puppeting. I just want the data from the several Discord communities I joined available through my Matrix server. And it's just me using it. But it should bridge the rooms properly and include the popular media formats, reactions etc.

"Alice has N brothers and she also has M sisters. How many sisters does Alice’s brother have?"

The problem has a light quiz style and is arguably no challenge for most adult humans and probably to some children.

The scientists posed varying versions of this simple problem to various State-Of-the-Art LLMs that claim strong reasoning capabilities. (GPT-3.5/4/4o , Claude 3 Opus, Gemini, Llama 2/3, Mistral and Mixtral, including very recent Dbrx and Command R+)

They observed a strong collapse of reasoning and inability to answer the simple question as formulated above across most of the tested models, despite claimed strong reasoning capabilities. Notable exceptions are Claude 3 Opus and GPT-4 that occasionally manage to provide correct responses.

This breakdown can be considered to be dramatic not only because it happens on such a seemingly simple problem, but also because models tend to express strong overconfidence in reporting their wrong solutions as correct, while often providing confabulations to additionally explain the provided final answer, mimicking reasoning-like tone but containing nonsensical arguments as backup for the equally nonsensical, wrong final answers.

• Arxiv, paper submitted on 4 Jun 2024
• PDF
• heise online article (German)

Hey, I'm fortunate enough to upgrade my home-server. I'd like to make some future-proof decisions. Which Matrix server do I choose? Dendrite? Conduit?

I like the candidate to be halfway well maintained, have active development during the next 4 years... Would be nice if it had a solid technological base and wouldn't hog that many resources. I'm okay if it still has some rough edges as long as they get ironed out in the near future.

It needs to provide service to me and a few friends and family. Audio calls would be nice and I definitely need it to connect to the Mautrix-WhatsApp/-Signal bridges.

Did I miss something? Is there another good server implemention apart from Synapse/Dendrite/Conduit?

Bonus questions:

1. Is Synapse the only server that can connect to SSO? Ideally I would like to maintain the user accounts via Authentik/LDAP/...whatever...
2. Is there a server that can handle multiple domains? Like an e-mail server that I can just tell "you handle mail for the following 5 domain names?"

I'm German and seems 'we' rely more on file hosters than torrenting. There are lots of tv series and movies with both the original audio track and the dubbed one on sites like funxd, serienjunkies, serienfans... They mostly redirect to a filecrypt.cc folder and then I get a DLC file to download the parts from turbobit or rapidgator (one-click hosters.)

What setup am I looking for, if I were to automate this? I'm aware of the Megathread but I didn't find the correct software to index those sites and then what kind of download manager people use nowadays. (Ah yes, and I don't want to pay for premium accounts.)

Edit: Replaced "one-click hosters" with "file hosters"

From the abstract: "Recent research, such as BitNet, is paving the way for a new era of 1-bit Large Language Models (LLMs). In this work, we introduce a 1-bit LLM variant, namely BitNet b1.58, in which every single parameter (or weight) of the LLM is ternary {-1, 0, 1}."

Would allow larger models with limited resources. However, this isn't a quantization method you can convert models to after the fact, Seems models need to be trained from scratch this way, and to this point they only went as far as 3B parameters. The paper isn't that long and seems they didn't release the models. It builds on the BitNet paper from October 2023.

"the matrix multiplication of BitNet only involves integer addition, which saves orders of energy cost for LLMs." (no floating point matrix multiplication necessary)

"1-bit LLMs have a much lower memory footprint from both a capacity and bandwidth standpoint"

Edit: Update: additional FAQ published

I'd like to play around a bit with an online shop. Nothing professional with proper requirements, just a hobby project. When googling for open source e-Commerce solutions, I can find the usual software. But I don't like open core models, and all the projects seem to want to make some money with an add-on marketplace. And most of the times the basic product seems very limited and they want you to extend it with proprietary extensions to get it usable in real-world scenarios.

Is there a project that does things differently? I mean for invoices I can choose between several platforms that won't push me to buy anything. I just can't find an online shop solution like that. My requirements would be something along: Sells products and keeps track of remaining stock, maybe sells services like online courses and software/pdf downloads. Can generate invoices and ties into payment providers. Maybe generates shipping labels. Isn't too bloated, a small, nice and clean hobby project will do. I'd like to avoid running a Wordpress/Drupal/Joomla underneath it if possible.

I get that companies have different requirements and commercial products are somewhat the obvious thing if you're doing commerce. But there has to be something aligned with the virtues of the free software community. Something I'd like to use to sell Tux stickers and power my Etsy shop with.

Today, OpenAI - the company behind ChatGPT - does their first developer conference. They said the keynote will be livestreamed at 10am PT.

I don't know if they have anything new to show off, but I thought I let you know as this is one of the major companies regarding AI.

Not sure if anyone has time to watch a conference in the morning or at noon on a monday, but it's going to be the evening in central europe where I live.

• https://devday.openai.com/
• YouTube Livestream

"This feature utilizes KV cache shifting to automatically remove old tokens from context and add new ones without requiring any reprocessing."

This means a major speed increase for people like me who rely on (slow) CPU inference (or big models). Consider a chatbot scenario and a long chat where old lines of dialogue need to be evicted from the context to stay within the (4096 token) context size. Previously the context had to be re-computed starting with the first changed/now missing token. This feature detects that, deletes the affected tokens from the KV cache and shifts the subsequent tokens in the KV cache so it can be re-used. Avoiding a computationally expensive re-calculation.

This is probably also more or less related to recent advancements like Streaming-LLM

This won't help once text gets inserted "in the middle" or the prompt gets changed in another way. But I managed to connect KoboldCPP as a backend for SillyTavern/Oobabooga and now I'm able to have unlimited length conversations without waiting excessively, once the chat history hits max tokens and the frontend starts dropping text.

It's just a clever way to re-use the KV cache in one specific case. But I've wished for this for quite some time.

• KoboldCPP
• release notes v1.48

They are referencing this paper: LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset from September 30.

The paper itself provides some insight on how people use LLMs and the distribution of the different use-cases.

The researchers had a look at conversations with 25 LLMs. Data is collected from 210K unique IP addresses in the wild on their Vicuna demo and Chatbot Arena website.

Yesterday Mistral AI released a new language model called Mistral 7B. @justnasty@lemmy.kya.moe already posted the Sliding attention part here in LocalLLaMA, yesterday. But I think the model and the company behind that are even more noteworthy and the release of the model is worth it's own post.

Mistral 7B is not based on Llama. And they claim it outperforms Llama2 13B on all benchmarks (at it's size of 7B). It has additional coding abilities and a 8k sequence length. And it's released under the Apache 2.0 license. ~~So truly an 'open' model, usable without restrictions.~~ [Edit: Unfortunately I couldn't find the dataset or a paper. They call it 'open-weight'. So my conclusion regarding the open-ness might be a bit premature. We'll see.]

(It uses Grouped-query attention and Sliding Window Attention.)

Also worth to note: Mistral AI (the company) is based in Paris. They are one of the few big european AI startups and collected $113 million funding in June.

• Details are on Mistral AI's Announcement
• techcrunch news article including information about the company
• They released an base/foundation model and an instruction-tuned one on HuggingFace
• And llama.cpp is already compatible and GGUF versions out there.

I've tried it and it indeed looks promising. It certainly has features that distinguishes it from Llama. And I like the competition. Our world is currently completely dominated by Meta. And if it performs exceptionally well at its size, I hope people pick up on it and fine-tune it for all kinds of specific tasks. (The lack of a dataset and detail regarding the training could be a downside, though. These were not included in this initial release of the model.)

EDIT 2023-10-12: Paper released at: https://arxiv.org/abs/2310.06825 (But I'd say no new information in it, they mostly copied their announcement)

As of now, it is clear they don't want to publish any details about the training.

I might be a bit late to the party, but for those of you that like ERP and fiction writing:

Introducing Pygmalion-2

The people from Pygmalion have released a new model, usable for roleplaying, conversation and storywriting. It is based on Llama 2 and has been trained on SFW and NSFW roleplay, fictional stories and instruction following conversations. It is available in two sizes, 7b and 13b parameters. They're also releasing a mix with MythoMax-L2 called Mythalion 13B.

Furthermore they're (once again) announcing a website with character sharing and inference (later in october.)

For reference: Pygmalion-6b has been a well known dialogue model for (lewd) roleplay in the times before LLaMA. It had been followed up with an underwhelming successor based on LLaMA (Pygmalion-7b). In their new blogpost they promise to have improved with their new model.

(Personally, I'm curious how it performs compared to MythoMax. There aren't many models around, that excel at roleplay or have been designed specifically for that use case.)