this post was submitted on 23 Nov 2023
1 points (100.0% liked)

Emacs

385 readers
1 users here now

A community for the timeless and infinitely powerful editor. Want to see what Emacs is capable of?!

Get Emacs

Rules

  1. Posts should be emacs related
  2. Be kind please
  3. Yes, we already know: Google results for "emacs" and "vi" link to each other. We good.

Emacs Resources

Emacs Tutorials

Useful Emacs configuration files and distributions

Quick pain-saver tip

founded 2 years ago
MODERATORS
rglullis@communick.news
raphael@communick.news
1
Visualising data analysis in org-mode (alien.top)
submitted 2 years ago by akuszyk@alien.top to c/emacs@communick.news
8 comments fedilink hide all child comments
 

Hello r/emacs! First-time poster here!

I recently wanted to visualise some data in org-mode using a Python source block to generate an image, which was then rendered directly in the buffer.

It took my a couple of goes to work out how to do it.

Given some data like this:

#+begin_src bash :results output file :file data.njson
echo '{"name": "Spock", "editor": "Emacs"}
{"name": "James Kirk", "editor": "Vim"}
{"name": "Dr McCoy", "editor": "Vim"}
{"name": "Scotty", "editor": "Emacs"}
{"name": "Worf", "editor": "ed"}
{"name": "Geordi LaForge", "editor": "Emacs"}
{"name": "Data", "editor": "Emacs"}
{"name": "Jean-luc Picard", "editor": "VS Code"}
{"name": "Wesley Crusher", "editor": "VS Code"}
{"name": "William Riker", "editor": "Vim"}
'
#+end_src

A visualisation can be rendered as follows:

#+begin_src python :results output file :file usage.png
import pandas as pd
import seaborn as sns
import sys

df = pd.read_json("data.njson", lines=True)
axes = sns.histplot(df, x="editor")
axes.get_figure().savefig(sys.stdout.buffer)
#+end_src

The main trick here is to set the :results output file header argument to write the output to a file, and to save the figure to sys.stdout.buffer from Python.

I've written about this on my blog too, where you can see the [unsurprising] results of the analysis!

you are viewing a single comment's thread
view the rest of the comments
[–] publicvoit@alien.top 1 points 2 years ago (2 children)

Is there somebody here using Nix(OS) who could teach me how to call this using a Python virtualenv where I deliberately setup pandas + seaborn before (in a shell)?

[–] acow@alien.top 1 points 2 years ago (1 children)

I highly recommend nix-direnv and the emacs direnv package. For this example, I quickly threw together a flake using a minimal template,

{
  description = "Python environment for plotting with Seaborn";
  inputs = {
    nixpkgs.url = github:nixos/nixpkgs/nixpkgs-23.05-darwin;
    flake-utils.url = github:numtide/flake-utils;
  };

  outputs = { self, nixpkgs, flake-utils }: 
    flake-utils.lib.eachDefaultSystem (system: 
      let pkgs = import nixpkgs { inherit system; };
          python = pkgs.python3.withPackages (ps: [
            ps.pandas
            ps.seaborn
          ]);
      in {
        devShell = pkgs.mkShell {
          buildInputs = [ python ];
        };
      }
    );
}

Then created a file .envrc with the contents use flake in the same directory as the flake.nix file, and ran direnv allow to allow use of it. I then used this org mode file to test,

Example of plotting from this [[https://andykuszyk.github.io/2023-11-18-using-emacs-org-mode-as-a-jupyter-notebook.html][blog post]].

Some data to work with,

#+begin_src javascript :tangle data.njson
{"name": "Spock", "editor": "Emacs"}
{"name": "James Kirk", "editor": "Vim"}
{"name": "Dr McCoy", "editor": "Vim"}
{"name": "Scotty", "editor": "Emacs"}
{"name": "Worf", "editor": "ed"}
{"name": "Geordi LaForge", "editor": "Emacs"}
{"name": "Data", "editor": "Emacs"}
{"name": "Jean-luc Picard", "editor": "VS Code"}
{"name": "Wesley Crusher", "editor": "VS Code"}
{"name": "William Riker", "editor": "Vim"}
#+end_src

And now we plot,

#+begin_src python :results output file :file usage.png
import pandas as pd
import seaborn as sns
import sys

df = pd.read_json("data.njson", lines=True)
axes = sns.histplot(df, x="editor")
axes.get_figure().savefig(sys.stdout.buffer)
#+end_src

#+RESULTS:
[[file:usage.png]]
[–] publicvoit@alien.top 1 points 2 years ago

Okay, it seems I'm not even halfway there to run NixOS on my own ...

[–] acow@alien.top 1 points 2 years ago

I highly recommend nix-direnv along with the direnv emacs package. I do everything with flakes, so I'd have a flake.nix that defines a shell with the inputs I need, then I'd have a .envrc with the contents use flake in the same directory. With those in place, you may need to run direnv allow in that directory, and then you can edit a .org file in the same directory as the flake and the python environment will have what you need.

Here's a flake I quickly made using a minimal template.

{
  description = "Python environment for plotting with Seaborn";
  inputs = {
    nixpkgs.url = github:nixos/nixpkgs/nixpkgs-23.05-darwin;
    flake-utils.url = github:numtide/flake-utils;
  };

  outputs = { self, nixpkgs, flake-utils }: 
    flake-utils.lib.eachDefaultSystem (system: 
      let pkgs = import nixpkgs { inherit system; };
          python = pkgs.python3.withPackages (ps: [
            ps.pandas
            ps.seaborn
          ]);
      in {
        devShell = pkgs.mkShell {
          buildInputs = [ python ];
        };
      }
    );
}

And then here's the .org file I tested with,

Example of plotting from this [[https://andykuszyk.github.io/2023-11-18-using-emacs-org-mode-as-a-jupyter-notebook.html][blog post]].

Some data to work with,

#+begin_src javascript :tangle data.njson
{"name": "Spock", "editor": "Emacs"}
{"name": "James Kirk", "editor": "Vim"}
{"name": "Dr McCoy", "editor": "Vim"}
{"name": "Scotty", "editor": "Emacs"}
{"name": "Worf", "editor": "ed"}
{"name": "Geordi LaForge", "editor": "Emacs"}
{"name": "Data", "editor": "Emacs"}
{"name": "Jean-luc Picard", "editor": "VS Code"}
{"name": "Wesley Crusher", "editor": "VS Code"}
{"name": "William Riker", "editor": "Vim"}
#+end_src

And now we plot,

#+begin_src python :results output file :file usage.png
import pandas as pd
import seaborn as sns
import sys

df = pd.read_json("data.njson", lines=True)
axes = sns.histplot(df, x="editor")
axes.get_figure().savefig(sys.stdout.buffer)
#+end_src

#+RESULTS:
[[file:usage.png]]