Note: this was originally a comment I wrote on
Lemmy in answer to the question “what
type of problems do you solve using Lisp?”. The post got to be a bit too
long, and I am re-publishing it here as a proper blog post. I am also
including some of a post I wrote on
Mastodon which
touched on some of these same issues.
So to answer the question: I have known about Common Lisp and Scheme for
years, but only recently started using them. This is the story of the 3
Lisp dialects that I use.
Emacs Lisp
I use Emacs and Emacs Lisp to manage my tens of thousands of text files,
I write Emacs Lisp scripts to automate simple tasks like searching for
pieces of information, formatting it, and outputting it to a report that
I might publish on my blog or send in an e-mail. I also use Emacs to
help with data cleaning before running machine learning processes. Emacs
helps with navigating CSV and JSON files, it also is a really good
batch file
renamer.
Scheme
I have recently started using Guile
Scheme to do some personal
projects. I went with Guile over the myriad other Scheme dialects
because it is the implementation used for the Guix package
manager and operating system.
-
Also, there the Goblins,
which is a distributed object-capability programming
system is
officially supported on the Guile platform, and I have been really
wanting to write applications using this programming style ever
since I first learned about it.
-
Also, there is the G-Golf
foreign interface layer allows Guile to automatically use an C
library that implements the GObject Introspection interface. So
through Guile, like with Python, you can use any C code library used
to create of all native apps in the Gnome,
MATE,
Cinnamon, or (my personal
favorite) the Xfce desktop environments. This
potentially makes Guile a viable alternative to Python scripting
across all of those Linux desktop environments.
Of all the Lisp dialects, Scheme is my favorite, for a few reasons:
-
It is absolutely tiny. Guile is relatively large (not as big as
Common Lisp), but other implementations are unbelievably small. for
example the Chez Scheme “petite” interpreter is fully compliant with
the R5RS standard, and the executable is like 308 kilobytes on a
64-bit Linux computer system.
-
Hygienic macros with
syntax-case
-
Recursive
functions
over using the loop
macro
of Common Lisp. When writing algorithms, I personally find it easier
to reason about recursive functions than loops. Scheme also provides
me the ease-of-mind that comes with knowing the optimizing Scheme
compiler will ensure recursive loops will never overflow the stack.
-
Pattern
matching
is well supported by most Scheme implementation.
-
It is a "Lisp-1" system, meaning there is only one namespaces
for variables and functions, as opposed to Common Lisp (a "Lisp-2
system") which allows a name to be either a variable, a function, or
both. I personally find it easier to reason about higher-order
functions
in Lisp-1 systems.
-
Support for Delimited
Continuations,
which is a fairly recent discovery of computer language theory
(first being discussed back in the 1990s), but is available across a
few Scheme implementations.
Common Lisp
That said, I am also starting experimenting with Embedded Common Lisp
(ECL) because it is a lightweight
standards compliant Common Lisp implementation that compile your program
into C++ code, and this is useful to my professional work.
The modern software industry, especially in the realm of big data and
machine learning, has mostly settled on a pattern of using C++ for
creating performance critical libraries, and creating Python binding to
the C++ libraries for scripting. I was hoping languages like Haskell
and/or Rust might come along and change all this, but it will take
decades (if ever) for the software industry to turn in that direction.
The problem with Python, in my experience (and I believe many other
software engineers would agree) is that it does not scale well to larger
applications at all, whereas Common Lisp does. This is for various
reasons, but mostly due to how Lisp does strong dynamic typing, and also
the CLOS implementation of the meta-object protocol. Yet too many
companies waste time writing large applications in Python — applications
that are much larger than the scripting use cases that Python was
originally intended to be used. I believe this is time and money better
spent on other things.
So I see Common Lisp, and the ECL compiler, as a potentially viable
alternative to the sub-optimal status quo of Python as a scripting layer
around C++ code libraries, at least perhaps for my day job, if not being
more generally true industry-wide. Mostly, ECL would allow me to write a
program in Common Lisp instead of Python, but deliver to my clients the
C++ code that ECL generates to be used in their machine learning
projects. (I have not actually done this yet, I am still investigating
whether this would be a viable solution to any of my projects).
ECL makes it easy to use C++ libraries through Lisp instead of Python.
And there are so many good C++ libraries out there:
Qt, OpenCV,
Tensorflow,
PyTorch,
OpenSceneGraph,
FreeCAD, Godot game
engine, Blender.
And it compiles easily on Linux/Unix (GCC), Windows (MSVC), and MacOS
(via Clang++), so good for cross-platform development.
Conclusions
So in spite of Lisp being such an old family of languages (its earliest
incarnations dating all the way back to 1958), and being superseded in
popularity and widespread use by languages like Python and JavaScript
across the software industry, Lisp is still a modern, relevant,
evolving, and very useful family of programming languages. At the same
time, a Lisp such as Scheme or Common Lisp would even be a better choice
of programming language in many applications where Python is currently
used.
I just hope I eventually find the time to try out all of these Common
Lisp and Scheme related ideas I have. I especially hope ECL turns out to
be a profitable technological choice for the professional work that I
do. But only time will tell.
Please feel free to comment here, or on Mastodon
I can explain the difference between X11 and Xorg with an analogy to the web and web browsers: X11 is like HTTP, Xorg is like the Chrome browser. X11 is the protocol, Xorg is software that implements that protocol.
X11 is old, it was designed back in the 1980s and includes messages for drawing lines and circles and fonts on the screen. Also, back then there were a lot of "thin clients", computers that were basically nothing but a browser, since graphics were computationally expensive and could not be done on the client computer, graphics rendering was done server side. There are lots of messages in the protocol for handling screen updates over a computer network.
Nowadays, all personal computers are powerful enough to render their own graphics, and no one needs the display server to draw individual lines or circles on screen. Vector graphics and fonts are done at the application level, not over the network. So these these messages specified in the X11 protocol are hardly ever used. Really, most of X11 (let's say 90% of it) is not used at all, only the parts where the keyboard and mouse are defined, and how you can allocate memory to buffer a graphic and copy that buffer to the display. But you still need to maintain the Xorg software to handle everything that X11 specifies, and this is just a waste of code, and a waste of time for the code maintainers.
So basically, they decided about 10-15 years ago that since no one uses most of X11, let's just define a new protocol (called Wayland) that only has the parts of X11 that everyone still uses, and get rid of the 90% of it that no one ever uses. Also, the protocol design takes into account the fact that most modern computers do all of their own rendering rather than calling out to a server to render for them. Also the Wayland protocol design takes into account that a lot of computers have graphics cards for accelerated graphics rendering.
Since the Wayland protocol is much simpler, it is easier for anyone to write their own software which implements the protocol, these software are called "compositors." Finally, 10 years after some of the first implementations of Wayland, the protocol and compositors are becoming mature enough that they can be used in ordinary consumer PCs.