Most of the artifacts you see are actually asteroids. We don't take all the colors at once when making an image. The ccds just count photons that fall on them, so we have giant glass filters that allow specific wavelengths through. We then image the fields multiple times with multiple filters and then map the wavelengths of light into color images (see other post about trying to make them as perceptually correct as possible). However, because taking multiple images over time means that things which are changing (i.e. moving asteroids) not not appear at the same places when we combine them. Most of the time these time changing effects get filtered out in the combination step, but sometimes if they are bright enough, or persistent enough the make it through.
As an FYI, you can right click in the sky viewer and it will show the coordinates of the object, and there is a button to copy a shareable link that will take anyone right to where you are looking in the sky.
Thank you for this useful reply. I appreciate the description of image building using the glass filters. That makes sense to me. The asteroids seem like an imaging issue that would be difficult to deal with since they are likely near-field objects that are in motion relative to far-field objects. That would also explain some of the trails that one sees if the objects along that trail are the same object imaged over a multi-step process.
I figured there had to be a way to get real coordinates so that people could do research but I was not bright enough to push the buttons to find that link and had to use the tool I hate - Snip & Sketch. I have a license for SnagIt and Greenshot on another workstation, either of which is better than S&S but that installation is inaccessible due to hardware issues.
Thanks for helping to make it easier to use these images!
If you click on the search icon, there are some quick links to fields across the sky so you don't need to scroll as much. There are some interface tweaks and ui stuff that will be coming down stream that will help with some of this as well.
I’m the person in the chain that takes the scientific data and maps it into these color images used in the viewer (among other responsibilities). I spent a long time researching and developing the software that as close as possible maps the data into human perception. It means so much to see people out there appreciating our work, and following what Rubin will achieve. So that you for taking your time to check it out!
Hey! Former astronomer here; I used to work a lot with SDSS data products. Do you happen to have a link to any sort of photometry DB/catalog for Rubin? Basically I’m looking to grab a ton of data but I’ve been out of the field for a while and I’m not sure where they publish their data sets and which ones you’d recommend as being relatively accessible.
You want to check out the early science release info at https://rubinobservatory.org/for-scientists/resources/early-... . This is all early science from preliminary data. It is mostly to help people ramp up developing their software against the type of things they will be seeing in the future. There some known issues and gotchas outlined in the release. Hopefully this will be enough to whet your appetite. The data seen in this sky viewer is part of the Data Preview 2.
I hope the data releases are a bit better than what I've seen from recent telescopes.
The common pattern seems to be that the majority of data is locked away behind multi-year embargo periods, or never released to the public at all with no explanation why, with only the smallest little crumbs of raw data actually released to the public, and usually years out of date. Also you have to go through some kafkaesque chain of technologies/servers/bureaucracy to get access to that little bit of data.
It's a little depressing to me when the public funds these telescopes and then get snubbed when it comes to getting access to the data they produce.
We are getting pretty far outside my area of expertise / knowledge on this topic here, but I know I and others agree with you here. There is a Rubin Observatory data access policy out there. Don't quote me on the exact specifics but I believe that most colleges and research institutions already have a mechanism to enable data access setup, or can get one. This is mostly an id auth type thing to make sure those accessing have data rights. There is a policy where people not associated with those can request access too, but I am not sure the procedure and would not be qualified to weigh in on anything more specific.
And whether it impacts your work? For example, if the unnamed agencies' efforts leave a telltale mark in areas they change, that would still be problematic, so presumably they have to place noise corresponding to adjacent pixels. Are you concerned about how data integrity affects your work output? Penny to understand the internal scuttlebut / any internal grousings about this.
Jaw-dropping stuff, looking forward to pointing 9-year old daughter (in "Future Astrophysicist" t-shirt) at it!
Possibly a naive question, after surfing the image for a while I didn't find any einstein rings (or arcs) - are they just rare/wrong scale or something else?
They are both fairly rare, and or like you said wrong scale (for this data release at least). There is actually one in this field, but it is below the noise floor of this image, it is very small and very faint. There will be others in the future that will be easier to see. By the end of our survey we will have about 40,000 sq deg (a bit over half the night sky) looking like what you see here.
Be sure you and your daughter click the search icon, there are a few other fields of stuff you can check out. The links in the search will take you there without needing to scroll around the big black sphere.
Thank you for the amazing work! Do you have any publications or articles describing your findings and the process you developed for mapping the scientific data into perceptually meaningful colour images?
Rendering our observatory's images in true HDR is something I've been wanting to implement for some time (especially as HDR support on the web continues to improve), as there is so much more in the data than we typically get to see.
I have a publication that will be done soon™. I have been preparing it for a while, but I keep finding edge cases to fix, and ways to make things even better that I keep changing things fast enough that I need to re-work the paper. The code to produce it will all be open source, and I am even working on making it an independent package you can install separately from the rest of our software stack, specifically so other places can easily install and use it in all sorts of environments.
It's funny you mention HDR, I am _just_ about finished with the HDR extension to the image processing code. What you see in the sky viewer is all standard SDR formats. As you suspect it makes a HUGE difference. Once this is all done we will be working on rolling out many different ways to experience it. The HDR code path will also be part of the package and is not much different to use than the standard stuff.
Reading your comment has gotten me a little more energized to make more progress on this this morning.
You are not wholly wrong! There is both a supporting structure for the mirror, AND a glass lens in front of the sensor to further flatten the incoming light.
The interesting thing about the spikes in our images is that they stay fixed in image plane coordinates, not sky coordinates. So as the night sky moves (earth rotates) the spikes rotate relative to the sky leading to a star burst pattern over multiple exposures.
Image creator here. We do dark field subtraction, as well as many other instrumental calibrations. What you are seeing is the fundamental photon noise. Because it is statistical in nature, you can never completely eliminate it. We could have chosen to put the black point in the image at a much higher flux level, but if you go to a high enough signal to noise level that you see no grain anywhere, you would miss out on so many interesting things that are still quite obvious to make out but are only 2-3 sigma above the noise.
Image creator here. Now imagine, when the survey is done, we will be able to see even fainter objects and image an area of the sky 1000x times this size.
Image creator here. This is such a massive dataset, most of the image processing needed to be custom written software pipelines. It not really practical for every pixel to be hand inspected. A few defects (and bright asteroids) imprinted through. It really hard to decide what is a real weird thing in the universe, and what is some sort of instrumental effect. We try to not pre-decide on what we think we should be seeing and filter for those by using things such as using classifiers. That leaves us with heuristics based on temporal information, size (is it smaller than a point spread function), and other related things. On large numbers of objects and pixels 1 in a thousand or 1 in a million outliers are bound to occur.
I'm glad you responded (i'm assuming you knew i wasn't criticizing the effort, but just in case -- I wasn't). I was assuming asteroid trail, but I've read that green stars can't exist and _could_ be a technosignature of "little green men". :) Your work on this is lovely. The combined effort of so many smart people over decades of work is truly heartening. Thank you.
I agree their results are also great! We do go a bit deeper, but he big difference it the speed we are able to build these images. We are able to image a larger area of the sky in each exposure, and are able to collect more light. This will lets us build images like this one in a few hours of observation, and build up an equivalent image of the entire southern hemisphere.
I'm the Rubin team member responsible for mapping the data into RGB images. I have been a long time reader of hacker news, but finally made an account to comment on this. I wanted to thank everyone here for their interest and taking their time to check out these images. Seeing everyone interested and engaged makes all the long hours worth it.
What range of wavelengths are in the original images? Do you produce multiple RGB images for looking at different things? c'mon, what does that entail? ;-)
The filters used for this range from near infrared to near uv. We used 4 different filters in all (for this image, the telescope has more). In general yes to fully appreciate all the color information as a human we need to generate different color combos so our eyes can pick up different contrasts.
However, what we strive for is being accurate to "if your eyes COULD see like this, it would look like this". To the best our our ability of course. We did a lot of research into human perception to create this and tired to map the information of color and intensity in a similar way to how your brain constructs that information into an image.
Let me tell you, I did not appreciate how deep a topic this was before starting, and how limited our file formats and electronic reproduction capabilities are for this. The data has such a range of information (in color and intensity) it is hard to encode into existing formats that most people are able to display. I really want to spend some time to do this in modern HDR (true HDR, not tone-mapping) where the brightness can actually be encoded separately than just RGB values. The documentation on these (several competing) formats is a bit all over the place though.
Edit:
I wanted to edit to add, if anyone reading this is an expert in HDR formats and or processing, I'd live to pick your brain a bit!
I'm impressed so much thought went into how to colorize the image! Sometimes it seems like space photos are just colorized thoughtlessly, or to increase the "wow" factor, so it's great to hear how careful and thoughtful you guys were in mapping this data to color-space.
As an FYI, you can right click in the sky viewer and it will show the coordinates of the object, and there is a button to copy a shareable link that will take anyone right to where you are looking in the sky.