Sign in

You might have come across various URL Shortening service providers, and it can be a problem going around and automating the entire url expanding task in your data.

Well, I found just the right library to do that.

URLEXPAND

This is a very well written and only available open source rust crate, which you can use to expand those silly short urls.

Example Usage:

use std::time::Duration;
use urlexpand::unshorten;

fn main() {
let url = "https://bit.ly/3alqLKi";
assert!(unshorten(url, Some(Duration::from_secs(10))).is_some()); // with timeout
assert!(unshorten(url, None).is_some()); // without timeout
}

Running your program:

cargo run --example unshorten

Current list of URL Shortening services supported:


Working in a startup that has a realtime SIEM tool as a product is a arduous task. While designing a realtime SIEM our team was mainly focusing on building data pipelines, handling large amounts of data, and scaling.

But little did we know about the problems that lied ahead which was testing out our SIEM tool with real-time logs that have some real threat context. And as every early-stage start-up, we did not have the luxury of data sets like big giants like Splunk might have.

We did not have logs to test out our correlations, train our ML models.


Well, first things first you will need Elasticsearch curator installed on your system.

I prefer installing it using pip as its easy peezy lemon squeezy. Just do

pip install elasticsearch-curator

There are other insane ways of installing it like using apt you can go to apt curator install instructions and follow the instructions there.

Now, the easy part. Follow along with the steps:

mkdir -p /elasticsearchData/es-backup
chown -R elasticsearch:elasticsearch /elasticsearchData/es-backup

sky

Software Developer | Backend Developer

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store