Jump to content

Nasa's Unsolved Big 10.7 Solar Flux Problem/ Unbelievable but True: Still NOT resolved


Recommended Posts

4 years ago I published massive faults in the NASA data files. It is not in the abstract, so you need to read the article:

https://link.springer.com/article/10.1007/s10509-020-03800-x

One of the reasons was that I found dozens and dozens of faults in the NASA database from the 10.7 flux. 

Main problem:

A big flare gives a false reading... and NASA puts it as correct

For instance on June 1, 2024 there was a flare and NASA published the enhanced reading: 233.3!

 

Originally it was even worse! The old data base gave monthly numbers. However, in the beginning they had sometimes only 25 measurements... What did they do? They divided by a monthly number!

For instance the old data base gave 131.19 while it was 138.5 for July 1950!

131,19 138,5 -5,3 1950 07 

 

Was there ever made an update and send an alert to all astronomers?

Nope... So astronomers check your old data base...

 

If you think NASA corrected all these faults...

NOPE... they are still on it...

@Hadeweka This is the way you should reply to my findings...

Link to comment
Share on other sites

8 minutes ago, Patrick P.A. Geryl said:

4 years ago I published massive faults in the NASA data files. It is not in the abstract, so you need to read the article:

https://link.springer.com/article/10.1007/s10509-020-03800-x

The final article is not openly accessible from that link, so I will refer to this version instead:

https://www.researchgate.net/publication/340477337_A_Formula_for_the_Start_of_a_New_Sunspot_Cycle

If I understand this correctly, it appears you are claiming that the old F10.7 source from Ottawa omitted some data in their calculations for the average values and that F10.7 data are generally distorted by solar flares?

I don't know too much about how the data processing works exactly there (and quite frankly, I don't have the time to look into the details), but the deviations seem relatively small to me, especially considering other error sources.

2 hours ago, Patrick P.A. Geryl said:

A big flare gives a false reading... and NASA puts it as correct

The flares just give outliers from the "regular" value. This doesn't mean that the values are false. In fact, I would argue that they are more "correct" than any adjusted values. They're just less usable for actual statistical purposes.

2 hours ago, Patrick P.A. Geryl said:

If you think NASA corrected all these faults...

NOPE... they are still on it...

That indicates malicious intent. Please refrain from such assumptions.

Up until this part, I don't see an issue with discussing the differences between F10.7 values, although I question if it's really worth the time - and I think you should use a more down-to-earth tone instead of accusing data manipulation.

 

However, in the remaining parts of the paper, there are some things about the paper that bother me.

Quote

Our new formulas make it possible that the sunspot number can be easily calculated from the 10.7 cm solar flux.

This was possible before, similar relations are known since at least 1972 [Stewart and Lefkin, 1972]. Your formula is essentially a linear variant of that formula with a lower gradient (potentially related to the way the F10.7 data are smoothed). This isn't a new result at all.

You do mention that there is a power law formula from [Johnson, 2011], but sadly you omit the fact that they also compare said power law with linear and polynomial models (e.g. a generally acknowledged model from the IPS in Australia). I miss the comparisons with your linear formula and these polynomial models.

Essentially, you just throw in a formula and present it as a brand new way to calculate F10.7 values, without even comparing it to older models. I have to be honest, I'm surprised that this paper passed peer review.

Because there's another issue:

Quote

5.1 Formula to find the start of a new sunspot cycle from mean fitted values

((F10.7/SSNf *100-100)/10)+100

This isn't a formula, it's missing a relational operator and physical units. While the sunspot number is unitless, the F10.7 isn't. You repeatedly omit the "sfu" in the paper. Also, you don't write what this value is supposed to be.

Inferring from the later parts, it's apparently a value to measure the strength between the fitted F10.7 values and the fitted sunspot numbers. I don't understand why you use this weird complicated relation instead of sticking to F10.7/SSNf instead (or rather the inverse, since spotless periods blow the value into infinity).

More importantly, why do you use the fitted values here? You are sacrificing SO MUCH accuracy by dividing two fits by each other instead of fitting the ratio AFTER calculating it. Why?

Quote

But what about the differences between the proxies at the low values? Why do they differ so much? After studying multiple possibilities, it seems that a hidden formula can be found: a usable predictor for the start of a new sunspot cycle.

Maybe they differ so much because these fits (and subsequently their errors) diverge highly for low sunspot values? Did you consider that possibility? If so, where are your calculations for the error propagation?

There's also no reason for the additional coefficients at all (and they're all without a unit). Firstly, why not use something like (F10.7/SSNf*10 + 90 sfu)? Is this just for scaling? Then you STILL have to ALWAYS include the proper units in the graph (once again, how did this pass peer review?), especially if you're comparing values of entirely different units (SSNf and F10.7).

Additionally, I don't see any discussion on how this value indicates a new solar cycle. You just write that it does, but don't SPECIFY it.

Quote

To our surprise we found a remarkable relationship between the monthly mean adjusted flux, the monthly mean sunspot number, and the start of a new sunspot cycle. For instance, if we divide the fitted adjusted flux with the fitted sunspots, then we found the strength of the difference.

I don't see anything surprising about this. The last sentence especially is somewhat of a tautology (except it's not a difference but a ratio). What did you expect to find? Gummy bears?

Quote

It shows us a usable predictor for the start of a new sunspot cycle, and delivers as an argument that the start of Cycle 25 lies somewhere in-between August 2019 and January 2020. The very low values in October 2019, with 27 values lower than 66 sfu and 4 below 64 sfu, provides us with a strong argument that this is the most likely candidate month for the start of Cycle 25.

You somewhat contradict yourself here. First you state that you found a usable predictor, only to revert to F10.7 values for your prediction again. Also the prediction isn't even useful, considering that the paper was published in 2020 and the time range for the cycle start was already predicted earlier to similar accuracy by the NOAA. You even write about that prediction:

Quote

The start of Cycle 25 in October 2019 would be a surprise for the scientific community because the NOAA Space Weather Prediction Center and NASA updated their preliminary prediction for Solar Cycle 25 onset on December 9, 2019: the current consensus is April 2020.

And they weren't wrong with either prediction. Because you omitted their error range of 6 months.

This paper is severly flawed in its methodology. I know you wrote it, but I would rather choose other sources for the F10.7 value discrepancy.

 

2 hours ago, Patrick P.A. Geryl said:

@Hadeweka This is the way you should reply to my findings...

I don't owe you anything, especially not answering you in a specific way that suits you. If you post things like these openly, expect to be criticised in different ways you might not even expect.

If you see this kind of criticism as hostility, you should do something else with your time, honestly.

8 minutes ago, Marcel de Bont said:

I have no idea what this is about and I am not going to make an account/buy/whatever to read an article. Who has more information, is there any truth to this or are we in silly land again?

Maybe to summarize it a bit, since my post got longer than initially expected:

The discrepancy between different F10.7 sources seems to be there, but it's kind of low. Could be worth investigating, but it won't really make a difference.

The rest of the posts are the massively flawed (yet for some reason peer-reviewed) paper and unjustified accusations towards NASA of data manipulation.

  • Like 3
Link to comment
Share on other sites

2 hours ago, Patrick P.A. Geryl said:

4 years ago I published massive faults in the NASA data files. It is not in the abstract, so you need to read the article:

https://link.springer.com/article/10.1007/s10509-020-03800-x

One of the reasons was that I found dozens and dozens of faults in the NASA database from the 10.7 flux. 

Main problem:

A big flare gives a false reading... and NASA puts it as correct

For instance on June 1, 2024 there was a flare and NASA published the enhanced reading: 233.3!

 

Originally it was even worse! The old data base gave monthly numbers. However, in the beginning they had sometimes only 25 measurements... What did they do? They divided by a monthly number!

For instance the old data base gave 131.19 while it was 138.5 for July 1950!

131,19 138,5 -5,3 1950 07 

 

Was there ever made an update and send an alert to all astronomers?

Nope... So astronomers check your old data base...

 

If you think NASA corrected all these faults...

NOPE... they are still on it...

@Hadeweka This is the way you should reply to my findings...

ok and what's the point

  • Thanks 1
Link to comment
Share on other sites

3 hours ago, Patrick P.A. Geryl said:

A big flare gives a false reading... and NASA puts it as correct

Not necessarily a problem at all. It could even be argued that including the times when flares happen at the same time gives a more accurate view of the long-term F10.7, since flares do in fact increase the F10.7 temporarily. Think about all the flares that happen throughout the day during active parts of the cycle, they all contribute to a higher F10.7, so the idea that you somehow need to adjust it down isn't a given at all.

I can understand why Jan would do it, i.e. to get a better view of the background F10.7, but there's nothing inherently "better" about it, or anything "wrong" about including flares in the readings. If we had a higher sampling rate, for example once every minute like with the X-ray flux, then you'd start to see how nonsensical it would be to remove flares; just like with the X-ray flux there's the idea of a background flux, which is also published, but it's not somehow "superior" to the actual flux.

3 hours ago, Patrick P.A. Geryl said:

The old data base gave monthly numbers. However, in the beginning they had sometimes only 25 measurements... What did they do? They divided by a monthly number!

For instance the old data base gave 131.19 while it was 138.5 for July 1950!

131,19 138,5 -5,3 1950 07 

This would indeed represent an error, and you even called that to my attention when I was wondering about why the values around the SC19 F10.7 minimum were so low; that is and was however very easy to correct for, and only really a problem during the very earliest parts of the F10.7 measurements, which is why I only saw the issue around that minimum.

Not sure what exactly the point of the thread is though, as these matters have already been discussed and resolved elsewhere several times. They are certainly not "massive faults"; the latter is a minor error that is easily corrected, and the former isn't even inherently an error at all.

At least you're shifting your focus back on the F10.7, which I think is a good direction for you to go in. Why not take whatever you want to discuss about it to the old F10.7 thread? It should still be going as far as I'm aware, and there's plenty of interesting discussion there, much of which has actual scientific merit (like e.g. the slightly unusual ratio of F10.7 to SN so far this cycle).

  • Like 4
Link to comment
Share on other sites

6 minutes ago, Philalethes said:

Why not take whatever you want to discuss about it to the old F10.7 thread? It should still be going as far as I'm aware, and there's plenty of interesting discussion there, much of which has actual scientific merit (like e.g. the slightly unusual ratio of F10.7 to SN so far this cycle).

I didn't even realize this was already discussed.

So essentially this whole thread is just a repost?

  • Like 2
Link to comment
Share on other sites

What if it could be proven that the parametric amplifiers weren’t properly affixed to the Wurzburg antenna farm? Or the waveguides were inaccurately calculated???  What would be the point of all this speculation. What is its goal precisely???

Edited by hamateur 1953
Link to comment
Share on other sites

9 minutes ago, Hadeweka said:

I didn't even realize this was already discussed.

So essentially this whole thread is just a repost?

More or less. It's been discussed both in that thread and in other threads too, like the "Growth of Cycle 25" thread. The minor errors in the early data Patrick points to has little to do with it though, and the differences between Jan Alvestad's manually corrected "background F10.7" and the measured F10.7 is also fairly minor in the long run; regardless of whether you account for or disregard these it appears that the ratio of F10.7 to 365-day average SN has been higher than usual so far this cycle. The exact reason for this can be speculated about, and I've provided some suggestions, but it ultimately lies within the range of what I'd expect natural variation in that ratio to possibly be like; if you look at the linear regressions on page two of that thread you can see that e.g. there are also times when the SN is high and the F10.7 is relatively that much lower than expected instead.

There are also gems in the thread like the adamant prediction of an SSN peak in May of 2023, heh. I'd suggest taking further discussion about it there in any case.

  • Like 3
Link to comment
Share on other sites

Thank you as always @Philalethes Mike 

8 minutes ago, Philalethes said:

More or less. It's been discussed both in that thread and in other threads too, like the "Growth of Cycle 25" thread. The minor errors in the early data Patrick points to has little to do with it though, and the differences between Jan Alvestad's manually corrected "background F10.7" and the measured F10.7 is also fairly minor in the long run; regardless of whether you account for or disregard these it appears that the ratio of F10.7 to 365-day average SN has been higher than usual so far this cycle. The exact reason for this can be speculated about, and I've provided some suggestions, but it ultimately lies within the range of what I'd expect natural variation in that ratio to possibly be like; if you look at the linear regressions on page two of that thread you can see that e.g. there are also times when the SN is high and the F10.7 is relatively that much lower than expected instead.

There are also gems in the thread like the adamant prediction of an SSN peak in May of 2023, heh. I'd suggest taking further discussion about it there in any case.

Only to reiterate this came up early in the cycle.  Both me and @3gMike spotted it independently I believe and discussed it briefly. Only as an apparent anomaly not related to active flare activities.  Also this ratio has definitely not remained consistent or predictable btw.  It is as can be seen in any of Jans charts graphically.  But we have been over this ad nauseum imo.  Mike 

  • Like 2
Link to comment
Share on other sites

8 minutes ago, Philalethes said:

There are also gems in the thread like the adamant prediction of an SSN peak in May of 2023, heh. I'd suggest taking further discussion about it there in any case.

Yeah, I just looked at that. It really aged badly, as well as his assumptions about a new historic activity minimum, while the Sun is now more active than in the last 23 years.

Any suggestions what happened there @Patrick P.A. Geryl?

Did you ever do a review of your prediction and the obviously contrary current data? Was there a follow-up paper from you?

  • Like 2
Link to comment
Share on other sites

Interestingly @helios solved something I had puzzled over regarding apparent missing data during a daily flux run.  The data isn’t missing at all.  Haha. It was my misinterpretation of the data that was the real issue.  And as far as NASA goes, like who cares?  The flare data? 

really Patrick.  🙄

Edited by hamateur 1953
Tired
Link to comment
Share on other sites

I only stumble into the forum seldom these days, but when I do, it's usually been to spotlight a critical intellectual failure, a glitch, or something very interesting in the universe.

I am pretty sure this post is one of these things. 

4 hours ago, Marcel de Bont said:

I have no idea what this is about and I am not going to make an account/buy/whatever to read an article. Who has more information, is there any truth to this or are we in silly land again?

I believe this is similar if not identical to my first ever response to Geryl's post on this website. Fortunately, there wasn't actually a paywall, I was just unfamiliar with the particular website(View PDF button on the side). Either way, this is a "coming full-circle" moment.

Geryl: The discovery of glitches in data should be forwarded to the stewards of said data immediately, in my opinion. It won't do them good if you're telling them about it here, and if you have corrections on top of that, I'm sure they'd be more receptive than we're able to be.

6 hours ago, Patrick P.A. Geryl said:

This is the way you should reply to my findings

On principle, I reject this call to social order. There is an existing range of valid responses within scientific or casual discourse which we can refer to, instead.

In other words, you can filter input on your end, but you cannot filter output of others.

Link to comment
Share on other sites

Posted (edited)

Nasa has still faults on its data base... Not the exorbitantly big ones from the past

For instance we published for 2005/08 92.8 instead of 100.7 a fault of only 8.5 percent

If I would make that error in school, I was disqualified

2005/09 we found 92.1 instead of 110.8  or 20.3 percent wrong

2006/12 : 6.9 percent

NASA never said to the people who had these values on their computer that they where wrong!

These values where used in peer reviewed articles! No peer reviewer ever objected!

A quick search for this cycle... Still, I would be disqualified in school...

2005/08 92.8 100.7 8.52005/09 92.1 110.8 20.32006/12 81.9 87.6 6.92011/03 114.6 122.9

2021 129  0 999.9????
2023  48  0 335.0
2023  56  0 273.7
2023 332  0 247.8
2024 153  0 233.3

 

Essentially, you just throw in a formula and present it as a brand new way to calculate F10.7 values, without even comparing it to older models. I have to be honest, I'm surprised that this paper passed peer review.

=> You are again unfair by overlooking that we compared to the BEST at that moment: Tapping and Morgan 2017.

 

Edited by Patrick P.A. Geryl
Link to comment
Share on other sites

6 minutes ago, UnderTheAnvil said:

What course would I have to take at University to understand all these words?

Space weather can certainly be a complex topic, and some knowledge of basic physics is definitely helpful, but through a combination of helpful articles here on SWL, search engines, and asking others if there are terms and/or concepts you still find confusing, you should be able to piece together what people are referring to in most cases. If there are any terms or statements you wonder what mean, don't hesitate to ask.

28 minutes ago, Patrick P.A. Geryl said:

Nasa has still faults on its data base... Not the exorbitantly big ones from the past

For instance we published for 2005/08 92.8 instead of 100.7 a fault of only 8.5 percent

If I would make that error in school, I was disqualified

2005/09 we found 92.1 instead of 110.8  or 20.3 percent wrong

The problem here is that you're relying on Jan Alvestad's manually corrected F10.7, which I criticized above. For example, on 2005-09-09 there was a recorded F10.7 of 717.6, presumably due to a flare. You can't tell NASA they are wrong for "correcting" that to a different value than Jan; either you acknowledge that the corrections are fairly arbitrary, or that there's a need for some standardized algorithm to determine the "background F10.7", but that's going to be very problematic at such a low cadence of measurements, you'd really need a much higher sampling rate for that.

The actual average F10.7 for that month (2005-09) based on the Penticton values, without attempting to correct for the flare, is actually 120.6.

  • Like 3
Link to comment
Share on other sites

46 minutes ago, Patrick P.A. Geryl said:

A quick search for this cycle... Still, I would be disqualified in school...

2005/08 92.8 100.7 8.52005/09 92.1 110.8 20.32006/12 81.9 87.6 6.92011/03 114.6 122.9

2021 129  0 999.9????
2023  48  0 335.0
2023  56  0 273.7
2023 332  0 247.8
2024 153  0 233.3

Absolutely, but you'd be disqualified for neither posting units nor descriptions of whatever these data are supposed to be. Without context, nobody would know the meaning of these numbers. And to be honest, I struggle as well. Are these F10.7 data from NASA? If so, where exactly did you get them from?

46 minutes ago, Patrick P.A. Geryl said:

For instance we published for 2005/08 92.8 instead of 100.7 a fault of only 8.5 percent

If I would make that error in school, I was disqualified

2005/09 we found 92.1 instead of 110.8  or 20.3 percent wrong

2006/12 : 6.9 percent

Same thing here, you just throw in a bunch of numbers of which only you know the exact context. This is not helpful.

And as Philalethes just wrote, just because datasets differ it doesn't mean that either of them is wrong. Often it's just the interpretation that is different (like F10.7 as it is measured, F10.7 with typical adjustments for atmospherical conditions and F10.7 with flares removed).

To me it sounds like one data set fits your later hypotheses better than the other one - and you therefore conclude that the other sets are wrong. However, as I explained to you multiple times now, you have to be EXTREMELY cautious with such assessments, as you might fall victim to confirmation bias.

46 minutes ago, Patrick P.A. Geryl said:

=> You are again unfair by overlooking that we compared to the BEST at that moment: Tapping and Morgan 2017.

It was the only comparison despite you referencing papers with other fits. Omitting their results wittingly is a form of scientific fraud.

 

Sadly you once again don't discuss my other points, so the discussion is over for me unless you finally do. Have a nice day.

Edited by Hadeweka
Typos and missing part about data
  • Like 2
Link to comment
Share on other sites

5 minutes ago, Hadeweka said:

Absolutely, but you'd be disqualified for neither posting units nor description of whatever these data are supposed to be. Without context, nobody would know the meaning of these numbers. And to be honest, I struggle as well. Are these F10.7 data from NASA? If so, where exactly did you get them from?

He should definitely provide references to exactly which data is being discussed, that's another long-running problem with these discussions for sure.

To be clear, it's from the F10.7 values measured at DRAO (Dominion Radio Astrophysical Observatory), typically referred to simply as "Penticton" due to its location near the city of Penticton; the values are published here. NRCan also has earlier F10.7 data dating back to 1947 accessible elsewhere. This data is what's almost universally used when it comes to analysis of the F10.7 over time.

In this case the number to the right of the year is the day number of the year, so day 48 would be February 17, where you can see in the above data that the distance-adjusted F10.7 is 335.0, much higher than the surrounding values due to a flare, which is what he's complaining about. I've already commented above on whether or not it's really more "correct" to manually change or remove that, so I won't go further into that at this point.

I can't really find the missing one from 2021 in the Penticton data though (999.9 means "missing" in the NASA files I've seen, from e.g. OMNIWeb), but it's not exactly something to make a huge fuzz about that data for a given date might be missing from an archive.

  • Like 3
  • Thanks 1
Link to comment
Share on other sites

9 hours ago, Patrick P.A. Geryl said:

Main problem:

A big flare gives a false reading... and NASA puts it as correct

For instance on June 1, 2024 there was a flare and NASA published the enhanced reading: 233.3!

Huh? Between 31 May and 2 June, the Penticton tri-daily 10.7cm readings were between 179 and 188, with 1 June's being 188, 188 and 184:  Daily Space Weather Indices dayind.txt (24.June01))

And a lot of times when it is flare enhanced, (like Jan) they will either note it is flare enhanced and/or use either the 17:00 or 23:00 reading.

On a side note, why is the 10.7cm/2800MHz reading exlusive to Penticton and not measured at Learmonth, San Vito or Palehua and vice versa?

Link to comment
Share on other sites

1 hour ago, Philalethes said:

He should definitely provide references to exactly which data is being discussed, that's another long-running problem with these discussions for sure.

To be clear, it's from the F10.7 values measured at DRAO (Dominion Radio Astrophysical Observatory), typically referred to simply as "Penticton" due to its location near the city of Penticton; the values are published here. NRCan also has earlier F10.7 data dating back to 1947 accessible elsewhere. This data is what's almost universally used when it comes to analysis of the F10.7 over time.

In this case the number to the right of the year is the day number of the year, so day 48 would be February 17, where you can see in the above data that the distance-adjusted F10.7 is 335.0, much higher than the surrounding values due to a flare, which is what he's complaining about. I've already commented above on whether or not it's really more "correct" to manually change or remove that, so I won't go further into that at this point.

I can't really find the missing one from 2021 in the Penticton data though (999.9 means "missing" in the NASA files I've seen, from e.g. OMNIWeb), but it's not exactly something to make a huge fuzz about that data for a given date might be missing from an archive.

Which date in 2021 @Philalethes I have almost everything recent easily available. Mike  btw @Patrick P.A. Geryl you can’t impress me by agreeing with me.  
Of course any large flare above a certain level may give a different reading if its waveform is within the 100 mhz slice that NRC monitors.  Haha. Its only been discussed here three times at the very least.  Please either hire a secretary or take notes.   Mike 

Edited by hamateur 1953
  • Like 1
Link to comment
Share on other sites

2 uren geleden, Hadeweka zei:

Absolutely, but you'd be disqualified for neither posting units nor descriptions of whatever these data are supposed to be. Without context, nobody would know the meaning of these numbers. And to be honest, I struggle as well. Are these F10.7 data from NASA? If so, where exactly did you get them from?

Same thing here, you just throw in a bunch of numbers of which only you know the exact context. This is not helpful.

And as Philalethes just wrote, just because datasets differ it doesn't mean that either of them is wrong. Often it's just the interpretation that is different (like F10.7 as it is measured, F10.7 with typical adjustments for atmospherical conditions and F10.7 with flares removed).

To me it sounds like one data set fits your later hypotheses better than the other one - and you therefore conclude that the other sets are wrong. However, as I explained to you multiple times now, you have to be EXTREMELY cautious with such assessments, as you might fall victim to confirmation bias.

It was the only comparison despite you referencing papers with other fits. Omitting their results wittingly is a form of scientific fraud.

 

Sadly you once again don't discuss my other points, so the discussion is over for me unless you finally do. Have a nice day.

The wrong data are still on Penticton! Our published faults were not corrected! Dozens and dozens of faults!

The correct values are on my researchgate link

https://www.spaceweather.gc.ca/forecast-prevision/solar-solaire/solarflux/sx-5-mavg-en.php

 

NASA published this link after my complaints... better but still a lot of faults

https://omniweb.gsfc.nasa.gov/form/dx1.html

Link to comment
Share on other sites

Guest
This topic is now closed to further replies.
×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue. By using this site, you also agree to our Terms of Use and our Privacy Policy.