AWS Ground Station Data Delivery To S3 Now Supported

MSCS and Engineering Thesis Topic Proposal

When I applied to the University of Washington’s MSCS and Engineering program in May of 2020, I had to submit a thesis topic proposal. Mine was based on using AWS’s Ground Station service to retrieve satellite data from Low Earth Orbit and Medium Earth Orbit satellites. Once I had retrieved and decoded the data from it’s original format, I wanted to use AWS’s SageMaker to use Classification modeling to detect areas where excavations of rare earth metals had occurred and the negative effects.

I did not get accepted into the program, but I did want to continue pursuing that research in attempt to get more familiar the technology and ultimately reach a conclusion.

In an effort to get more familiar with Machine Learning, I am going through ACloudGuru’s Machine Learning certification course.

Technologies used throughout the research:

AWS SageMaker and SageMaker GroundTruth (helpful for abeling data)
Amazon Mechanical Turk
Terminology: Supervised vs Unsupervised, Labeled vs Unlabeled, Structured vs Unstructured..Image data and classification..Databases, Data Warehouse (Unstructured and filtered in import), Data Lakes (Unstructured and unfiltered until export)

Data Stores used for storing data in AWS:

S3
RDS
DynamoDB
TimeStream
RedShift
DocumentDB

Data Pipeline is a great tool for moving data from one location to another. Can also be used for transforming the data. You can also manage the pipeline, the resources that transfer the data, and retry logic.

Streaming Data and Open Source Datasets

Kaggle is a great resource for finding large datasets for experimenting with
Kinesis Data Streams: AWS service for handling streaming data (receiving streaming data from Data Producers, like an IoT device, click-stream data, etc). A Kinesis Shard is what is used to carry or hold that data before sending it to AWS Consumers (which are used to process or store that data). Shards consist of a sequence of data records, as many as 1000/second. They are also referred to as Transient Data Stores, meaning the data held in the Shards is only held for 24 hours up to 7 days.
Data Preparation – usually the most time consuming process that includes transforming a dataset using different techniques to prepare it for model training.
Categorical Encoding – is the process of manipulating categorical variables when Machine Learning algorithms expect numerical values as inputs.

Learn how to take care of basil, mint, and other plants
Just got a new fish! His name is Sherman and he’s a Galexy Koi Beta
I began reading some fiction novels to keep my mind sharp and intact with my creative side
Studied and finally passed my Amazon Web Services Solutions Architect – Associate exam
Bought a new rug!
Applied (got accepted to some and rejected from others) to grad school programs around the country (MBA and Masters in Computer Science programs)
Got back into running for some consistent exercise (granted I need to do a better job of that nowadays!)
I also got into Facebook fights about politics and religion with friends family (Do NOT recommend doing this). Staying off social media more and more has been doing incredible things for my mental health. Even deleting the apps from the app store on my phone, but whenever I really want some down time and I’m bored I’ll sign in to my socials from a browser (getting constant notifications was a nightmare for me!)
Reach out and be available for newly graduated students or those that had lost their job due to Covid-19 related reasons.
Watched and re-watched the entire Harry Potter series over and over again! It’s so good, and unfortunately wasn’t allowed to watch the movies growing up.
Watched Ozark, Vampire Diaries, and many many more movies and series’ on Hulu, Netflix, Disney Plus, HBO Max (The Emperor’s New Groove is my favorite Disney classic!!)
Took part in my first hot wings challenge – this was TOUGH. I only made it through 5 of the 10 hot sauces!
Bleached part of my hair!!! I actually really enjoyed it too

There are plenty of other things that can help relieve stress and keep your mental positive. I highly encourage reading a new book (The Great Gatsby is my favorite, 1984 is really interesting, Slaughterhouse-five is a classic). Twilight is another good series of books too.

A few pictures of my rug, my fish, Sherman, and my plants –

So you feel like a phony sometimes?

It’s okay and totally normal to feel like a complete impostor sometimes. Everyone in the world feels like that every once in a while.

I remember thinking on my first day of my first full-time opportunity out of college, I really am not qualified to be here. I thought everyone around me knew Everything there was to know about everything in the world as it relates to professional and personal matters. Except me. That’s who I lacked confidence in in that moment. And it took leaving that job after nearly a year and a half to realize that I am good enough. Not just professionally, but personally. I am okay. I’m alright. And if need be, I can fake it until I make it with the best of em!

I did believe that.

It took taking some time away from friends and family, time away from working early mornings and late evenings to realize whatever I put my mind to and commit to, I can not only achieve, but thrive at.

And that doesn’t have to be your way of dealing with impostor syndrome either, I realize that. But step one in the process for me was voicing the words out loud, ‘I don’t think I’m good enough’, and ‘I’m scared’, and finally..’and that’s okay. I will figure this thing out’. And much later on, and for me a very important step was eventually voicing , ‘you can do this Dale, I believe in you’, as I’m looking at myself in the mirror.

Falling down can hurt. It can hurt your pride, it can hurt your self-esteem, it can make you feel like you just don’t measure up in life, whatever your current circumstances look like. Voicing that out loud, even if it’s just to yourself, may help you realize that in fact you are just a human. Hardly more evolved than a monkey. And I’ve seen a monkey eats their own shit before. So yeah, if you don’t eat your own shit, you’re already headed in the right direction.

But another part of falling down, is the process of getting back up. Whether that’s getting let go of a job, not delivering your responsibilities on time at work and maybe missing an important deadline, or just thinking on your first day on the job that you’re not cut from the same clothe as everyone else. But the people we think are succeeding beyond measure in life, they also put their pants on one leg at a time just like the rest of us. And most successful people haven’t always thought they would be successful. Most are afraid of losing the success and accumulated worth they’ve worked for like it could all be gone tomorrow. We’re all insecure, and we all fight some form of a battle. Mental, emotional, physical. You name it. No one has the secret formula for life, believe it or not.

Some things that help me?

Taking a deep breath
Taking a bite out of a lemon
Meditating for 30 minutes or longer
Yoga for 30 minutes or more to some soothing music on YouTube
Sometimes watching videos on animals in the wild just being goofy

Space –

Recently, I’ve been really interested in using AWS’s GroundStation service that launched in 2018. I’m going to also try and use this as an outlet for posting links, things I’m “researching”, and how I am getting more familiar with and using the service.

I love that space is becoming more of a realistic frontier, and less and less of an ethereal topic. With AWS’s Ground Station service, there is now broader accessibility for ‘commanding, controlling, and downlinking data from [Low Earth Orbit (LEO) and Medium Earth Orbit (MEO)] satellites.’, which is a direct quote from the services’ FAQ that I will post below. That sounds like pretty revolutionary stuff right there if you ask me.

The data coming from these satellites can be stores and streamed in EC2 instances and processed or later stored in another service like S3. In order to get started with the service, you can schedule a specified amount of time, ‘contact times’, to schedule access to a satellite. It’s recommended to follow these steps according to the services’ FAQ:

Schedule a Contact
Use the AWS Ground Station EC2 AMI (Amazon Machine Image) for the Contact: ‘a Command instance to receive operational telemetry from the satellite and transmit changes to the satellite’s planned future activities, and a Downlink instance to receive bulk mission data from the satellite’. This type of AMI will communicate with the AWS Ground Station antenna gateway via ‘an ENI (Elastic Network Interface) connection that exists between the EC2 instances and the satellite antenna for the duration of the ‘Contact’.
Make sure to have read the pricing for the types of ‘Contacts’ offered. Click here for a good resource on pricing.

It’s important to be well-read on the service offerings and the type of data you will receive from downlinking (here’s a great resource for what that means and more!) data from the satellites.

Resources mentioned in this article:

Ocean –

During the quarantine, I’ve been making the best of it by learning things about nature. Things like how to care for plants, but also swimming patterns of Whales from the arctic to the antarctic. Also like how Killer Whales..aren’t even whales! They’re Orcas, dolphins! And to compliment learning about satellites and how we communicate with them via parabolic satellites through radio waves. I’ve been really really fortunate to keep my job, I’m certainly in a privileged position to have kept my job and be able to better utilize the time I would have been spending commuting back and forth from the Whole Foods Market office.

Also, this may seem random, but Jesus Christ can we do something about this?? This team, the Ocean Cleanup Project looks like they’re doing some incredible work to clean up our planet’s oceans!! Big shout out and THANK YOU on behalf of all of us irresponsibly living citizens not paying attention to what’s happening where our trash is going and how it’s impacting our planet. I want to take better care of this planet and do my part on changing the way I contribute to the problem and rather draw attention to ways we can all do better and be aware of the current situation.

Ocean Gyres

Marine Traffic

Types of Antennas

Tech Talk: What I’ve been working on with Amazon Web Services lately –

First I’d like to talk about how I put this website together. I’ve really wanted a place where I could put my thoughts, but also a place where I could share the things I’ve been learning, studying, and practicing.

I’ll jump right into it. I signed into my Amazon Web Services console. Which, if you’d like to follow along, go ahead and sign up for an account here. When you create an account, you will be able to sign in to your ‘root’ account. For best-practices sake, go to AWS’s IAM (Identity and Access Management) service and create another user with your ‘root’ account user, store the credentials for the ‘root’ user account, enable multi-factor authentication on your ‘root’ account, and then going forward use the new user and that user credentials.

When you create the new user, you can provide that user with Administrative access via a role or adding that user to a Group (that has an Administrative Policy attached) to leverage all services from the AWS console and/or the AWS CLI (which I use often and love using! Check it out and download here – make sure the download is compatible for your computer/laptop/machine’s operating system).

Next, I went over to AWS’s EC2 (Elastic Compute Cloud) service. This service is incredible because you can create instances in a matter of minutes and stop and terminate them as soon as you are finished with them. You can also used a pre-configured template for the root instance called a publicly available AMI (Amazon Machine Image) that have things like PHP, MySQL, and WordPress installed at launch time.

I searched with the keyword ‘blog’ and selected the first option from the AWS Marketplace – a WordPress AMI Certified by Bitnami and Automattic.

WordPress

Once I specified the instance details I wanted, I reviewed and launched my instance. Once my instance had finished launching, I clicked on the instance id which led me to a dashboard with the instance details. I right clicked on my instance > Clicked on Instance Settings > then chose Get System Log. I scrolled toward the bottom of the Instance Logs and found the password I would need to log into my WordPress instance. I copied the password locally to a clipboard. I looked back at my running instance details on the Instance dashboard and found my instance’s IPv4 Public IP (example: 100.100.100.100). I went to my instance and logged in with my username and password and began customizing the site, the WordPress Theme, and adding content to each page! Finally, I went to Route53 and clicked Create Record Set, which allows me to map my IPv4 Public IP to a domain name. I had previously bought my domain name (daleyarborough.com) for about $14, so once I mapped the IP to my domain name, I cleared my browser cache (you may need to do this in order to see your website after it’s finished propagating), and voila I was able to see my site live!

Hope that helps if you are interested in trying to create your on WordPress site or spin up your first EC2 instance. Please make sure to read the AWS’s service before using them in order to avoid incurring large costs (my wordpress instance

Get more familiar with the services mentioned in this article by clicking the links below!

Related

By Dale Yarborough