David's Astronomy Pages Notes - Session 654 (2019-01-31)

Notes (S653)

Notes Main

Home Page

Notes (S655)

	Session Aims & Highlights
	- Observing Result  - Night Summary Plot  - Session Event Log
	Operational Issues
	- Critical Issues (1),  Major Issues (2),  Minor Issues (3),  Continuous Improvement (6)
	Investigation : Unexpected Dome Closure & Lack of Obs.Manager Response
	Images from 2019-01-31  >>
	Aurora Images/Video from 2019-01-31 >>

Session Aims & Highlights (2019-01-31)

Main aims

1. Acquire images of a range of targets (comets, variable stars, nearby stars and deep sky objects) during a night of expected very good weather conditions
   
2. Test AstroMain 3.8.5 program to confirm recent fixes working and check for any newly introduced issues
   
3. Continue confirmation of robust Observatory System (hardware & software combined)

 
Equipment & Software

•  12" LX200 and ST-10XME
• 80mm APO and ZWO ASI 178MC
• CCDApp2, TheSky6, CCDSoft5, PHD2,  POTH Hub
• Pegasus UPB & Motor Focus, TCF-S
   
• AstroMain 3.8.5 AstroPlan 1.6.2

Highlights

• Images.  Of the 60 targets on the nigjht's observing plan , x targets were successfully imaged
   
• Lock-In procedure worked well when a Net FrameWork Exception Window was unexpectedly found after remotely logging into the Observatory Computer at daybreak.   Screen captures were taken as each window / application was systematically revealed & examined. The check confirmed that the session had run to completion and shutdown procedures ( dome, telescope, camera)  had all completed ok.   The source of the Exception Window and its details were successfully captured.

Lowlights

• Unexpected Dome Closure. An unexpected / unhandled dome closure at 01:15 caused some lost observations, but thelost interval was partially cloudy anyway, and resulting investigation and actions will lead to improvements in future automated operations.

Observing Plan (2019-01-31, S654)

Observing Result (2019-01-31, S654)

Night Sky Summary Plot - 2019-01-31, S654 Top axis: Sky Brightness at Zenith (in ADU/s) Lefthand axis: Local Time (hh LT). Righthand axis: Sun Altitude (degs)

Session Event Log (2019-01-31, S654)

Time   Event Detail 17:48 Session Created Live Session Created (2019-01-31 S00654, ImageSaveNum: 654001) 17:50   Camera1 Connected SBIG Camera Connected (set point -30°C) 17:50   Telescope Connected Telescope Connected (TheSky6) 17:51 Observatory (Auto) Observatory placed in Fully-Automated Mode 17:51 Session Pending Session pending (2019-01-31) 17:51   Plan Requested Observing Plan requested from AstroPlan () 17:54   Plan Loaded Observing Plan loaded to queue (2019-01-31 V1 BEST PLAN, Plan ID: 261) 17:54   Camera1 Connected SBIG Camera Connected (set point -30°C) 17:54   Telescope Connected Telescope Connected (TheSky6) 17:55 Shutter Opened Shutter opened (opening time 45s) 17:55 Equilibration Started Dome Equilibration started 18:00 Equilibration Ended Dome Equilibration ended (5.3 mins total) 18:00   Queue Started Observing Queue started (60 targets selected) 18:00     Target Started (1/60) Target started (1/60, NGC 1022) 18:20     Target Completed Target completed (1/60, NGC 1022) 18:26 Focusing Started (Foc1) Foc1 Focusing Started (TCF-S) 18:29   Focusing Completed Foc1 AutoFocus Completed (Profile No 1) 18:40     Target Completed Target partially completed (2/60, NGC 1999) 18:45     Target Completed Target completed (3/60, NGC 1444) 18:51     Target Completed Target completed (4/60, NGC 1545) 18:54     Target Started (5/60) Target started (5/60, IC 443) 19:19     Target Completed Target completed (5/60, IC 443) 19:22     Target Started (6/60) Target started (6/60, NGC 1931) 19:38     Target Completed Target completed (6/60, NGC 1931) 19:40     Target Started (7/60) Target started (7/60, GCVS U Gem) 19:47     Target Completed Target completed (7/60, GCVS U Gem) 19:49     Target Started (8/60) Target started (8/60, GCVS DX Cnc) 19:57     Target Completed Target completed (8/60, GCVS DX Cnc) 19:59     Target Started (9/60) Target started (9/60, GCVS RZ LMi) 20:10     Target Completed Target completed (9/60, GCVS RZ LMi) 20:11 Critical Conditions Alert Critical Conditions Alert (Obs.Manager will close the Shutter 20:12     Target Aborted (10/60) Target aborted (10/60, NGC 1528) due to cloud 20:12 Session Suspended Session suspended 20:14 Session Resuming Session resuming 20:14 Queue Resumed Observing Queue resumed 20:14 Shutter Opened Shutter opened (opening time 5s) 20:20     Target Started (11/60) Target started (11/60, NGC 1647) 20:29     Target Completed Target completed (11/60, NGC 1647) 20:30     Target Started (12/60) Target started (12/60, GCVS NV Ori) 20:39     Target Completed Target completed (12/60, GCVS NV Ori) 20:47     Target Completed Target completed (13/60, GCVS CN Ori) 21:08     Target Completed Target completed (14/60, UGC 118) 21:16     Target Completed Target completed (15/60, GCVS Z And) 21:16     Target Started (16/60) Target started (16/60, GCVS RX And) 21:23     Target Completed Target completed (16/60, GCVS RX And) 21:24     Target Started (17/60) Target started (17/60, NGC 1535) 21:37     Target Completed Target completed (17/60, NGC 1535) 21:46     Target Completed Target completed (18/60, NGC 1817) 21:56     Target Completed Target completed (19/60, GCVS DY Per) 22:05     Target Completed Target completed (20/60, GSC 1224:29) 22:05     Target Started (21/60) Target started (21/60, NGC 1788) 22:22     Target Completed Target completed (21/60, NGC 1788) 22:34     Target Completed Target completed (22/60, NGC 2022) 22:35     Target Started (23/60) Target started (23/60, NGC 2186) 22:37 Focusing Started (Foc1) Foc1 Focusing Started (TCF-S) 22:41   Focusing Completed Foc1 AutoFocus Completed (Profile No 2) 22:49     Target Completed Target completed (23/60, NGC 2186) 22:57     Target Completed Target completed (24/60, GCVS FO Ori) 23:04     Target Completed Target completed (25/60, NGC 1980) 23:12     Target Completed Target completed (26/60, GCVS GK Per) 23:21     Target Completed Target completed (27/60, NGC 1342) 23:21     Target Started (28/60) Target started (28/60, NGC 1664) 23:31     Target Completed Target completed (28/60, NGC 1664) 23:31     Target Started (29/60) Target started (29/60, 46P/Wirtanen) 23:54     Target Completed Target completed (29/60, 46P/Wirtanen) 00:04     Target Completed Target partially completed (30/60, NGC 1513) 00:25     Target Completed Target completed (31/60, NGC 2628) 00:31     Target Completed Target completed (32/60, LAMOST-HVS1) 00:45     Target Completed Target completed (33/60, NGC 1027) 00:56     Target Completed Target partially completed (34/60, GCVS RR Tau) 00:56     Target Started (35/60) Target started (35/60, NGC 2126) 01:04     Target Completed Target partially completed (35/60, NGC 2126) 01:13     Target Completed Target completed (36/60, NGC 2281) 01:19     Target Started (38/60) Target started (38/60, 38P/Stephan-Oterma) 01:30     Target Completed Target completed (38/60, 38P/Stephan-Oterma) 01:39     Target Completed Target completed (39/60, GCVS SY Cnc) 01:50     Target Completed Target completed (40/60, C/2015 O1 (PANSTARRS)) 01:50     Target Started (41/60) Target started (41/60, GCVS AM CVn) 02:00     Target Completed Target completed (41/60, GCVS AM CVn) 02:00 Session Suspended Session suspended 02:19 Session Resuming Session resuming 02:19 Shutter Opened Shutter opened (opening time 45s) 02:20     Target Completed Target completed (42/60, IC 983) 02:21     Target Started (43/60) Target started (43/60, NGC 6181) 02:45     Target Completed Target completed (43/60, NGC 6181) 03:00     Target Completed Target completed (44/60, NGC 1907) 03:08     Target Completed Target completed (45/60, GCVS SU Tau) 03:21     Target Completed Target partially completed (46/60, GCVS U Her) 03:24     Target Started (48/60) Target started (48/60, GCVS R CrB) 03:31     Target Completed Target completed (48/60, GCVS R CrB) 03:31     Target Started (49/60) Target started (49/60, GCVS CN Leo) 03:46     Target Completed Target completed (49/60, GCVS CN Leo) 03:54     Target Completed Target completed (50/60, 61 Cyg) 04:01     Target Started (52/60) Target started (52/60, GCVS AM Her) 04:10     Target Completed Target completed (52/60, GCVS AM Her) 04:11     Target Started (53/60) Target started (53/60, GCVS CY Lyr) 04:20     Target Completed Target completed (53/60, GCVS CY Lyr) 04:21     Target Started (54/60) Target started (54/60, GCVS S Cyg) 04:36     Target Completed Target completed (54/60, GCVS S Cyg) 04:36     Target Started (55/60) Target started (55/60, GCVS SS Cyg) 04:45     Target Completed Target completed (55/60, GCVS SS Cyg) 04:49     Target Started (56/60) Target started (56/60, WDS GRB 34) 04:57     Target Completed Target completed (56/60, WDS GRB 34) 05:05     Target Completed Target completed (57/60, GCVS CI Cyg) 05:12     Target Started (59/60) Target started (59/60, GCVS EV Lac) 05:20     Target Completed Target completed (59/60, GCVS EV Lac) 05:20 Queue Completed Job Queue completed 05:20 Session Closing Session closing 05:21 Shutter Closed Shutter closed (closing time 50s) 05:31 Session Finished Session Finished

Back to Top

Operational Issues (2019-01-31, S654)

• Critical Issues (1)
  
• Major Issues (2)
  
• Minor Issues (3)
  
• Continuous Improvement (6)

[ Prev | Next ]

Critical Issues

• Obs.Manager didn't respond appropriately when dome shutter unexpectedly closed at 01:15. The Issue has been investigated (see notes "Investigation - Session Suspension Issues").   The Obs.Manger needs to recognise the unexpected shutter closure and intervene after 1 minute to reopen the Shutter if conditons are safe or to properly suspend the session if they are not.
  Fixed 2019-02-03 (AstroMain 3.8.7.  Obs.Manger should now recognise the unexpected shutter closure and intervene appropriately)

Major Issues

• AstroGuard program inappropriately closed the shutter immediately after program was opened . This was whilst conditions were safe and session was active.
  Closed (unable to ascertain the cause of the problem or a fix. AstroGuard has now moved on to a new version).
   
• Data corruption when saving Session Form in AstroPlan. Attempt to save manual changes to Session Form caused loss of information about completion status of each target on the form and database behind.  Unable to identify a root cause to the problem.   It is likely that database changes where overwritten by older values held on the Form.
  User should referesh form before making/saving required changes
  Closed (user awareness on probable cause and additional safeguards added to form for current session / plan )

Minor Issues

• Pegasus UPB software produced a Net Framework exception message associated with Environmental charts resource issue.  Issue didn't stop the session from running to completion, but it was an opportunity to test out new Lock-Down procedures.   Similar framework message appeared on xxxx
  Note sent to Pegasus Astro on 2019-02-05. Further note sent to Pegasus Astro on
   
• The 'CurrentFrame' field needs to be cleared after finising HFD Fast Focusing
  Fixed 2019-02-02 (AstroMain 3.8.6)
    
• Understand why target NGC 1528 failed.   During slew to target a Critical Conditions Alert was raised at 20:11 and the Obs.Manger proceeded to Abort the Target and the Suspend the Session at 20:12.  Almost immediately the Alert was lowered and the session was resumed at 20:14.   By this point execution had moved to the next item in the queue (NGC 1647) but this had to wait 6-8 minutes to start. (ASCOM Logs show that an CloseShutter Command was sent to the Dome at 20:12:44, with an OpenShutter command sent at 20:14:48).  There is no indication that the Dome responded to the Close Command. This is worrying...(Unexpected Dome Closures are one thing, but unexpected failure to Close are another).   There is no decrease in Clarity values at the time of the dome closure. In fact the Clarity values (av 48) are amoungst the highest seen.  Obs.Manager doesn't say the reason for the session suspension was cloud though later on the stated reason for tthe target abort was cloud. 
  Closed (unable to ascertain what caused the target failure).

Continuous Improvement

• Build a new routine that will process an entire session's worth of Raw FITS Folders. At the moment each Folder of Raw Fits Files has to be reduced individually by clicking a button to accept the reduction
  Done 2019-02-03 (AstroMain 3.8.7)
   
• Can CCDSoft's Camera Control be stopped from popping up after Copying Reduced Files. It's annoying. Camera Control windows pops up as a by-product of calling AbortAutoguide  in PostJobTasks
     Fixed 2019-02-10 (AstroMain 3.9.1, AbortAutoguide is now only called if Session is in Observatory Mode)
   
• Need to add UPB Env Temp difference to Obs Temp Difference Chart & Change Title. Also need to change title boxes names e.g 'Env Humidity' to 'UPB Env Humidity'
    Done 2019-02-02
   
• Change scale of Observatory Temperature difference from outside from +8/-4 to +4/-4
    Done 2019-02-02
   
• Redo NGC 1817 as a Mosaic. Mosaic prepared and added to Targets Database (AstroDatabase). Mosaic acquired  2019-09-08 (S691) (see NGC 1817 - Open Cluster (H VII-4) ).
  Done 2019-09-08 (Session)
   
• Examine correlation between RMS guide values and quality of associated image frames.  Desktop exercise looking at data and images. New tools have been developed to show guiding performance in terms of individual guiding charts and summary guiding data for each session.
  Closed (2020)

[ Prev | Next ]

Back to Top

Investigation : Unexpected Dome Closure & Lack of Obs.Manager Response (2019-01-31)

During an interval of cloud cover between 01:15 and 3:00,  the Dome was automatically closed. However the sesssion wasn't correctly suspended and slewing of scope & imaging of the darkened interior of the dome continued throughout.  See plot below.

Extract from NightSummary6 Plot showing the relevant event Dome Shutter Closed during Cloudy Interval but with Telescope Operations continuing

Time Line

Time line showing the order of key events is tabulated below

Graph of Sky Clarity measurements from Cloud Sensor vs Dome Events

Analysis

Ascom Logs confirm that the Dome Shutter closed at 01:20  (47s to Close),  and then opened again at 02:19 (48s to Open)

Whilst the Dome Closure was preceeded by an OpenShutter Command at 02:19:06, there was no preceediing CloseShutter Command when the Shutter was closed.  

(The Log File contains earlier CloseShutter Commands at 20:12 & 22:55 (issued by AstroGuard program in error)
and then a final CloseShutter Command at 05:20:47 (issued by AstroMain when the session was closed) , but no CloseShutter command at around 01:20)

This indicates that the Shutter Closure at 01:20 was not requested from the Observatory Computer via POTH.Dome  (ie not from AstroGuard/AstroMain/ POTH Window clients). Since there was no one in the Observatory to push the 'Red Button' on Shutter Drive Unit, the Shutter must have been shut as a result of a Relay trigger or a random glitch in the Dome Controller or the Shutter Drive Unit.

The Clarity levels dropped below a value of 35 at the time that the Shutter Closed.   This might indicate that a Relay Trigger from the CloudSensor Head might have led to the Shutter Closing.   However the CloudSensor Relay was believed to be set to 'Unsafe' when Clarity falls below 20, not 35.

A problem with the correct operating of the Relay has been noted before on  xx and again in daytime testing on xx and xx. There have also been issues with the Dome unexpectedly closing during recent sessions.

1) May be the Cloud Sensor Head figures in the Aurora Cloud Sensor III software are not actually being saved in the CloudSensor Head. However there is no configuration file where these values are stored. The Software has read them from the Head in the first place. 

2) Another possibility is that the Thresholds are being mis-interpreted.  It is noteworthy that with the Threshold value for Clear/Cloudy of 20 and Cloud/Very Cloudy of 15, that added together (20+15) one gets a value of 35, which is precisely the value that Sky Clarity dropped below when the Dome closed.

This looks like it could be the answer. However if I examine the unexpected Dome Closure Event during Session 2019-01-28 S651 (which was also typified by the absence of an immediately preceeding Dome Closure Command in the ASCOM log)  Clarity values at the time of the Dome Closure were around 45.

Extract from NightSummary6 Plot showing the relevant event Dome Shutter Closed during Cloudy Interval but with Telescope Operations continuing

Later Notes / Update:
Obs.Manger Code was later modified so the the state of the weather was checked after an Unexpected Shutter Closure and the shutter reopened if conditions were ok or suspended the session if they weren't.

The problem of unexpected shutter closures was resolved later by setting the Delay in Dome Controller's Relay Settings from 0s to 30s.

Back to Top