608 private links
Excellent page listing their feeds, and not only their published articles but also separate feeds for submissions and announcements.
Code shared on https://git.scipost.org, although many repos are sparse or have not been updated in years.
They also have their own Mastodon instance, with an account for each field.
All in all, very impressive work by what appears to be primarily Dutch academics.
First heard about it from 4 gravitons.
Thanks to git log -1 it is trivial to view the latest commit in a repo, and combined with find ... -exec ... we can easily do that for multiple repos at once (for example see my commits bash function).
Turns out doing the same for the earliest commit in each repo is not at all trivial. Here is my approach (it should be said up-front that it's not pretty). But I wanted to get a handle on when I started working on some repos, and this pulled out the data I needed without having to do it manually for each individual repo.
taha@asks2:/media/bay/taha/projects/ansible/pub/roles
$ find . -maxdepth 2 -name ".git" -exec sh -c "git -C {} --no-pager log --all --pretty=format:\"%cs %h%d %s [%cn]\" | tail -n1" \; -exec printf " %s\n" {} \; | sort -n
2021-03-14 02506d4 Initial commit [taha] ./texlive/.git
2021-03-14 14ddd9a Initial commit [taha] ./desktop-tools/.git
2021-03-14 2bfe5bc Initial commit [taha] ./R/.git
2021-03-15 0c2deda Initial commit [taha] ./calibre-web/.git
2021-04-02 7a2bdb2 Initial commit [taha] ./iriun-webcam/.git
2021-04-13 167ef2d First commit [taha@asks2] ./shaarli/.git
2021-05-28 ca70e06 Initial commit [taha] ./wallabag/.git
2021-06-09 3d4891e Initial commit [taha] ./lxd-server/.gitThe directory roles contains a bunch of git repos (and possibly also a few folders that are not repos, which we exclude from find with -name ".git". Then for each repo, we list all commits (across all branches) using a custom format but then, importantly, only showing the last one which is the earliest commit.
To be able to tail this git command (and not the entire find command) we need to put it inside sh -c ....
Then we use a second exec statement to append the current directory name (so we can orient ourselves in the output) and then sort the entirety on the output (which thanks to our custom git format lists the commit date in the first column). Note that piping strips the colour from the git output, so no need to set colours in our pretty-format.
- https://stackoverflow.com/questions/51376361/getting-commit-history-from-multiple-repositories-for-some-period
- https://stackoverflow.com/questions/5188914/how-to-show-the-first-commit-by-git-log (this approach with
git rev-list --max-parentsdid not work in my case, not sure why) - https://andrewrea.co.uk/posts/git-log-over-multiple-repos
- https://stackoverflow.com/questions/50882822/view-logs-of-multiple-git-repositories
- https://stackoverflow.com/questions/6712423/how-do-i-count-the-number-of-commits-made-in-to-all-repos-hosted-by-gitosis-on-m
- https://stackoverflow.com/questions/49388069/what-does-git-log-1-do
- https://stackoverflow.com/questions/307015/how-do-i-include-a-pipe-in-my-linux-find-exec-command
- https://stackoverflow.com/questions/1371261/get-current-directory-or-folder-name-without-the-full-path
The collection spans from digitized copies of eighteenth century journals through the latest Open Access conference proceedings and preprints crawled from the World Wide Web.
Month-by-month moonsighting reports from around the world.
- Signal app and protocol (end-to-end encrypted by default, but centralized server, no federation) - use it daily
- Matrix protocol (end-to-end encrypted, decentralized, federated, user gets to choose app to use) - finds it use, but can be slightly annoying when using multiple devices
- Delta Chat - please see notes below
- Briar - does not rely at all on a central server, so can work even when Internet is down (via wifi, BT or memory cards). Syncs via Tor if Internet is up, which does make the mobile app a battery hog. I have not tested this personally, thanks to Slim for insights
- Quicksy (an XMPP/Jabber client). But as for XMPP, consider that although it's federated, it's not very private, e.g., discussion here.
- Simplex Chat - I haven't tested this
Matrix servers infinitely accumulate user data in the transport layer, namely encrypted messages and attachments as well as cleartext metadata about users and groups. A permanent metadata and room membership record may be ok, or even useful accountability-wise, for government officials but is not suitable for many people threatened by rising authoritarianism where “guilt by association” suffices for prosecution or worse. Operators of servers who can access all the social metadata about its users, and maybe even public messages, are becoming themselves a potential target for hacking or authoritarian repression.
Signal’s key architectural innovations were ephemeral message routing, usable E2EE with metadata-minimization where the transport server only stores messages temporarily and aims to know nothing about anything (except your phone number). Metadata-minimization is why Signal’s founder felt fine to entrust all “Signal Server” activities to Google, Amazon, Cloudflare, Microsoft, our very fine riders of the big tech apocalypse. A big step for cryptography in 2014 but a daring gamble, looking from 2025.
chatmail relays follow the Signal transport model and use SMTP and IMAP only for ephemeral message transport where the server keeps literally nothing except a (random) password and last login time. Turns out, if you hold those ancient protocols right, you can have a metadata-minimizing e-mail transport server with 0,001 EUR of costs per year per address.
https://lobste.rs/s/muk05v/reflections_on_matrix_criticism_over#c_1sijfo
Delta Chat
From the FAQ:
- Images, videos, files, voice messages etc. are handled as usual.
- For performance, images are optimized and sent at a smaller size by default, but you can send it as a “file” to preserve the original.
⚡️ Sign up to secure fast chatmail servers or use classic e-mail servers
- Chatmail relays Github repo - Chatmail relays are designed to be very cheap to run, and are automatically deployed and updated. Official list of relays.
- Besides the recommended usage of chatmail relays, you can use many classic email accounts in chatmail apps. List off email providers known to work with Chatmail.
- Chatmail core Github repo
List of clients is quite impressive, includes Android, iOS, a special "family-geared" Android client, a TUI client, and more.
Push notifications
Instant push notifications available for iOS and Android devices.
They seem to have put careful thought into how to handle notifications, especially on Android. By default, depends on Google FCM Push service or microG.
But you can change this to instead use a "background connection" or a "forced background connection". I have not yet found any client that supports UnifiedPush.
Note that classic e-mail servers do not support Push Notifications for Delta Chat users.
Audio or video calls
Not really, at least not yet.
You can invite others to an audio/video chat, but it's essentially a Jitsi meeting (or whatever provider you choose). No ring tone on the other end, so you must have otherwise arranged the "call".
- https://delta.chat/en/help#how-can-i-use-audiovideo-calls-with-delta-chat
- https://support.delta.chat/t/i-made-a-video-call-app-with-15-second-ping
- https://github.com/WofWca/video-call-over-email
- https://github.com/WofWca/webxdc-video-call
webxdc apps
Watch out for insecure copy-cats
As far as I can tell these services, also named ChatMail, have nothing whatsoever to do with Delta Chat.
Delta Chat setup flow
I would like to test using the same profile on multiple devices.
So on device 1 (phone) I downloaded and installed the Delta Chat app from the F-Droid appstore. Opened the app and followed the instructions to create my profile. That was painless.
On my second device (tablet) I downloaded and installed the ArcaneChat app from F-Droid. Opened the app, clicked on "Add second device" on the welcome screen which starts the camera viewfinder. On my already registered device (phone), I opened the app, opened Settings, and clicked "Add Second Device". (Note! Don't make the mistake of clicking the QR code icon next to "Settings", because that will not work for adding secondary device, only for inviting others to a conversation).
ArcaneChat TUI client
I chose to create ~/.local/git/arcanechat, set it up for python using direnv echo "layout python3 > .envrc", then installed the client using pip:
$ pip install -U arcanechat-tui[full]This installs the arcanechat-tui binary, an alias arcanechat, as well as the deltachat-rpc-server in .direnv/python-3.10/bin/.
Next, how to configure the existing account?
- https://github.com/ArcaneChat/arcanechat-tui/issues/91 - Setup as second device
- https://github.com/ArcaneChat/arcanechat-tui/issues/96 - How to add account as second device?
Create a backup in ArcaneChat Android and transfer it to the computer.
$ arcanechat import <path-to-backup-tar>Worked flawlessly with a single profile in the first device.
Medfarm Play är en videoportal som skapats i samarbete mellan pedagoger och MedfarmDoIT.
Här kan du hitta tusetals undervisningsfilmer, många av dem indelade i ämnesspecifika kanaler.
Video recordings on many subjects from Uppsala University, such as Nobel lectures, doctoral conferment ceremonies, etc. Use the search function or browse the channels.
Let's say you are looking for old Ansible code dealing with multiple distros (meaning multiple files in the ./vars/ directory) to draw inspiration from. How can we quickly identify the roles with vars directories containing more than a single file?
$ cd /media/bay/taha/projects/ansible && find . -not -path "*/archived/*" -not -path "*/testing/*" -path "*/vars/*" -printf "%h\n" | sort | uniq -d && cd $OLDPWD
./pub/roles/apache/vars
./pub/roles/common-systools/vars
./pub/roles/desktop-environment/vars
./pub/roles/digikam/vars
./pub/roles/graphics-driver-nvidia/vars
./pub/roles/php/vars
./pub/roles/php-versions/vars
./pub/roles/python2/vars
./pub/roles/python3/vars
./pub/roles/R/vars
./pub/roles/sioyek-pdf/vars
./pub/roles/wallabag/varsAll of the returned paths indeed contain at least two files.
Note how we limited find to only directories named vars.
The Research Software Directory is designed to show the impact research software has on research and society. We stimulate the reuse of research software and encourage proper citation of research software to ensure researchers and RSEs get credit for their work.
Arabic language library for Python, provides basic functions to manipulate Arabic letters and text
Note to self: please come back to this post with links on R, its packages and related resources.
Grammar of graphics
- https://github.com/tidyverse/ggplot2
- https://exts.ggplot2.tidyverse.org/gallery - community-maintained list of extensions
- https://ggplot2-extended-book.com - how to use ggplot2 extensions
- ggiraph - makes ggplot2 graphics interactive.
- ggstraw - custom ggplot2 geom for deviations. Via https://nacnudus.github.io/duncangarmonsway/posts/2019-01-25-a-custom-ggplot2-geom-for-deviations.
- ggpattern - geoms with pattern fills that can be mostly anything (patterns, parts of images, etc.).
- ggside - add side panels to various geoms.
Other visualization frameworks
- https://grantmcdermott.com/tinyplot - tinyplot, an awesome extension of the base R graphics system.
- lattice
Both
ggplot2andlatticeare built on top of the lower-levelgridpackage, which was incorporated into the base R distribution back in R 2.0.0.
https://grantmcdermott.com/tinyplot/#fnref1
Pipes, pipes, pipes
- https://nathaneastwood.github.io/2020/02/01/get-and-set-list-elements-with-magrittr - a neat way to get and set list elements in magrittr pipes thanks to some abuse of the
.Primitivefunction. Could offer a nice way to handle higher-dimensional data.
Integration with reference manager Zotero
- https://github.com/oeysan/c2z (manipulate your Zotero collections, items from R; can be integrated with other tools such as cronR, mailR, or even HomeAssistant)
PDF tools
- https://ropensci.org/blog/2016/03/01/pdftools-and-jeroen (pdftools - A fast and portable PDF extractor)
Integration with GPG
- https://ropensci.org/technotes/2016/10/19/gpg-release (Encryption and Digital Signatures in R using GPG)
Web scraping
- https://blog.rsquaredacademy.com/web-scraping Introduction to web scraping with
rvest, 2019 - https://www.brodrigues.co/blog/2018-11-01-nethack Scraping with
rvestand building a data package, 2018 - My own package periodicdata uses
rvestto create a data package
Email from R
- https://github.com/rstudio/blastula send great-looking HTML email messages from R
- https://github.com/datawookie/emayili send email messages from R
- https://github.com/rpremraj/mailR utility to send emails from R
Integration with MS Office (yes, yes, I know...)
- https://ardata-fr.github.io/officeverse
- https://github.com/davidgohel/officer
- https://github.com/davidgohel/flextable
Interfacing with Google Docs
Package repositories
- https://cran.r-project.org/web/packages/available_packages_by_name.html
- https://www.cranhaven.org - for recently archived CRAN packages
Resources
- https://emilyriederer.netlify.app/post/team-of-packages (post of Emily Riederer's rstudio::global 2020 talk)
- https://indrajeetpatil.github.io/awesome-r-pkgtools Awesome R Package Development Tools, compiled by Indrajeet Patil, 2024
- https://win-vector.com/2017/02/05/evolving-r-tools-and-practices John Mount, 2017
- https://github.com/nanxstats/awesome-shiny-extensions
- https://zenodo.org/record/7023492#.YwnwHuxBxhH R from Zero to Hero, slides from talk by Batool Almarzouq (in Arabic)
- https://r-packages.io/datasets - list of R CRAN packages with public datasets
Tools to consider
- https://alexioannides.com/2016/11/02/asynchronous-and-distributed-programming-in-r-with-the-future-package (the
futurespackage: distributed computation, non-blocking async input/output, and more) - https://cran.r-project.org/web/packages/geomtextpath/vignettes/geomtextpath.html
The Open Knowledge Network connects specialists of the open movement and promotes them through the Open Knowledge Global Directory.
CKAN is an open-source DMS (data management system) for powering data hubs and data portals. CKAN makes it easy to publish, share and use data.
CKAN is a free open source Data Management System (DMS) to build open data portals, and its code is held in trust by the Open Knowledge Foundation.
This is a one-liner that I find useful when deciding a free IPv4 address to assign a new container:
$ lxc network list-leases lxdbr0 | grep STATIC | sort -t "." -n -k4
| taipei | 00:16:3e:9c:dd:28 | 10.252.116.3 | STATIC |
| hunan | 00:16:3e:f7:4f:b2 | 10.252.116.4 | STATIC |
| xian | 00:16:3e:b0:54:12 | 10.252.116.5 | STATIC |
| samarkand | 00:16:3e:e8:79:60 | 10.252.116.6 | STATIC |
| karachi | 00:16:3e:e5:4c:47 | 10.252.116.7 | STATIC |
| lahore | 00:16:3e:f1:73:ab | 10.252.116.8 | STATIC |
| muscat | 00:16:3e:9a:89:05 | 10.252.116.9 | STATIC |
| antioch | 00:16:3e:92:79:ca | 10.252.116.11 | STATIC |
| delhi | 00:16:3e:22:9c:d0 | 10.252.116.13 | STATIC |
| bahrain | 00:16:3e:4b:b6:ee | 10.252.116.14 | STATIC |
| lusail | 00:16:3e:fd:1f:49 | 10.252.116.15 | STATIC |
| pergamon | 00:16:3e:0e:9c:82 | 10.252.116.16 | STATIC |The one-liner is simplified by the fact that only the IP address field contains dots, so we can use them as delimiters for sort without first having to sort on the column delimiters.
Non-spherical nanostructures explain the fruit’s colour.
I never reflected on the colour of blueberries, and just assumed it was pigment-based. Turns out it is more interesting than that!
R Middleton et al, Sci. Adv., 2024, https://dx.doi.org/10.1126/sciadv.adk4219
I suppose this should be seen in light of the official endorsement of Matrix by eSam over a year ago, which I noted at the time.
Sweden’s public sector is keen to adopt open source and open standards to ensure data sovereignty and protection against vendor lock-in
As a part of Tele2 Collaborate, Element is seen as a natural Skype for Business (S4B) replacement for Sweden’s public sector, and an alternative to US-based Big Tech vendors.
Stöd
- https://www.slutarokalinjen.se (samtal eller chatt på svenska, även samtal på arabiska)
- https://www.1177.se/Stockholm/liv--halsa/tobak-och-alkohol/tobak/hjalp-att-sluta-roka
- https://www.1177.se/Stockholm/undersokning-behandling/behandling-med-lakemedel/lakemedel-utifran-diagnos/lakemedel-mot-rokning
Kunskap
Keep code, data, containers under control with git and git-annex
Built in Python.
- https://doi.org/10.21105/joss.03262 (paper)
- https://github.com/datalad/datalad (repo)
- https://www.datalad.org
- http://docs.datalad.org (docs)
- http://handbook.datalad.org (crash course)
Best way to write and share your knowledge in markdown.
Pretty neat!
You do not need to master every aspect of git to make use of it in your daily work.
Even a little bit of git will take you a long way towards best practice with regards to reproducibility!
Enjoy these videos
- https://youtu.be/s3JldKoA0zw (science setting, also awesome soundtrack!)
- https://www.youtube.com/watch?v=CvbLVVRzJF8 (business setting)
Git introductions or tutorials
- The missing semester of your CS education, online classes on the command-line, Git, the shell and more from MIT
- Git Magic, by Ben Lynn
- Become a Git pro in just one blog. A thorough guide to Git architecture and command line interface, by Uday Hiwarale
- How To Make Life Easier When Using Git, by Shane Hudson
- Confusing git terminology, Julia Evans, 2023-11-01
- How Git Works!, Julia Evans, 2024-06-03
- Inside .git, Julia Evans, 2024
- Git en välskriven guide på svenska från IT-institutionen vid Uppsala universitet
- Beej's Guide to Git, very thorough tutorial, offered in various HTML and PDF formats. Via https://links.kalvn.net/shaare/0n4qig.
- https://ohmygit.org - An open source game about learning Git!
- https://ohmygit.blinry.org - Oh My Git 2.0 in your browser. Via https://chaos.social/@blinry/111011979500389143.
Opinions or comments
- How to be a 'good' git evangelist?, by Sunniva Indrehus, 2021-09-02
- Git is my buddy: Effective Git as a solo developer, Mikkel Paulson, 2021. Via https://www.eamoncaddigan.net/posts/git-links
- Nobody cares about your Git history, Dan Kelch, 2024
- https://matklad.github.io/2023/12/31/git-things.html
- Nordic RSE
- RSSE Africa
- RSE Asia Association rse-asia.github.io
Countries
- Germany: de-rse.org
- Netherlands: Community of Research Software Engineers in the Netherlands
- Belgium: be-rse.org
- UK: Society of RSE Engineering
- France: DEVLOG French RSEs
- USA: us-rse.org
- AUS/NZ: RSE Association of Australia and New Zealand
- Canadian Research Software Community
- Swiss RSE
Links and notes
Via Github.
- FAIR principles for Research Software, released in May 2022 as an endorsed recommendation of the Research Data Alliance
- FAIR software checklist - if you are a researcher who writes software, check it out! Via Netherlands e-Science Center
- Research Software Engineers International
- RSE Worldwide: Opportunities to Strengthen the Global RSE Community - ReSA, Sep 2024. With notes on the state of country-based RSE associations.