David's Astronomy Pages Notes - Session 1046 (2022-10-13)

Notes (S1045)

Notes Main

Home Page

Notes (S1047)

	Session Aims & Highlights
	- Observing Result  - Night Summary Plot  - Session Event Log
	Operational Issues
	- Critical Issues (0),  Major Issues (2),  Minor Issues (3),  Small Defects (0),  Continuous Improvement (2)
	Images from 2022-10-13 >>        [ Local Files >> ]

Session Aims & Highlights (2022-10-13)

Main aims

1. Targets.  Acquire images of a selection of variable stars, nearby stars, comets & deep sky targets as allowed by time & twilight conditions.

 Equipment & Software

• 12" LX200 and ST-10XME
• 80mm APO and ZWO ASI 178MC
• CCDApp2, TheSky6, CCDSoft5, DeviceHub 6.6.0.16,  MeadeGeneric 1.3.5.414, UPB 1.5, ASCOM 6.6
• Pegasus UPB & Motor Focus, TCF-S
• SharpCap 3.2, PHD2  2.6.11dev2
   
• AstroMain 3.55.3,  AstroPlan 1.32.5,   AstroShCap 1.3.12, AstroGuard 1.9
  AstroVOE 1.12  AstroSN 1.8.6,  AstroProtect 1.7,  AstroMove 1.9.1
  (AstroWeather 1.18.1, AstroAllSky 3.17.1,  AstroObsCam 1.9, AstroLaunch 1.9.4)

Highlights

• Targets.  Data was acquired from only 4 out of 30 planned targets, with partial data acquired from 1-2 further targets (one of which was later rejected due to bad guiding).  This was due a bug with the handling of Rain Trim setting at start of session and later on due to cloud.

Notes:

• Start of session was delayed by a problem with the application of Rain Trim Setting. This also caused the first 3 targets to be missed/skipped.
• Cloud (high cloud that reduced star count without dropping Cloud Sensor's Clarity value effected the session significantly and it was not until later 04:04 that sky was clear enough for productive observing.
  

Summary Plots & Logs

Observing Plan

Observing Result

Dome & Scope Slewing Performance

Slew/Centering Performance

Guiding Performance

Sky Conditions (Locate Frames)

Night Sky Summary Plot Top axis: Sky Brightness at Zenith (in ADU/s) Lefthand axis: Local Time (hh LT). Righthand axis: Sun Altitude (degs)

Actual Weather vs Pre-Session Weather Forecast

Session Event Log

Time     Event Detail 21:01:19 Session Monitoring AutoStart monitoring for Live Session opportunity between 21:01 & 05:39 21:01:21 Session AutoStarting Session autostarting (21:01) 21:01:54 Session Created Session Created (Live, 2022-10-13 S01046, ImageSaveNum: 1046001) 21:01:56   Scope Switched On Telescope Power has been switched on via UPB Powerbox. 21:03:39   Services Started Observatory Services started 21:03:46 Observatory (Auto) Observatory placed in Fully-Automated Mode 21:03:51 Session Pending Session pending (2022-10-13) 21:03:53 Session Initiating Session initiating (2022-10-13) 21:03:55   Plan Requested Observing Plan requested from AstroPlan (1.32.5) 21:04:36   Plan Loaded Observing Plan loaded to queue (Plan ID: 808) 21:04:56   Camera1 Connected SBIG Camera connected (set point -10°C) 21:05:03   Telescope Connected Telescope connected (TheSky6) 21:05:27 Session Equilibration Session ready to Open Dome 21:06:42   Dome Opened Dome opened (opening time 45s) 21:06:58 Session Running Session running 21:07:00   Queue Started Observing Queue started (32 targets selected) 21:07:03     Target Started (NrZen) Target started (Focus Field 23, HIP 113729) 21:07:17   Dome Unparked Dome unparked 21:10:55       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S) 21:13:24       Focusing Completed Foc1 AutoFocus Completed (Profile No 1, wide) 21:15:22       Focusing Completed Foc1 AutoFocus Completed (Profile No 1) 21:15:24       Focusing Started-Foc2 Foc2 Focusing Started (Secondary Scope, using ShCap) 21:17:30       Focusing Completed Foc2 AutoFocus Completed (Profile No 2, wide) 21:19:12       Focusing Completed Foc2 AutoFocus Completed (Profile No 2) 21:19:29     Target Completed (NrZen) Target completed (Focus Field 23, HIP 113729) 21:20:23     Target Started (1/32) Target started (1/32, AT2022why) 21:23:57       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S) 21:26:09       Focusing Completed Foc1 AutoFocus Completed (Profile No 3) 21:32:29     Target Completed Target completed (1/32, AT2022why) 21:32:33     Target Started (2/32) Target started (2/32, WDS GRB 34) 21:33:58 Obs.Monitor Frozen Obs.Monitor appears to be frozen at around 21:33 in section Update Foc1 Fields 21:38:42 Job Queue Frozen Job Queue appears to have frozen at around 21:38 - 21:38 21:41:47   SoftSuspend Called Soft Suspend is called due to Deteriorating Conditions (too few stars) 21:43:07 Obs.Manager Obs.Manager is frozen (last cycle was at 21:43:07) 21:43:07   Critical Cloud Alert Critical Cloud Alert (Obs.Manager will close the Dome) 21:43:09   HardSuspend Called Hard Suspend is called due to Critical Conditions (cloud) 21:46:46   CCDSoft Restarting CCDSoft being killed and restarted in attempt to fix locked threads 21:57:00 User Intervention User Intervention to restart frozen ObsManager Program 21:58:10   Services Stopped Observatory Services stopped 21:59:18 Observatory (Manual) Observatory placed in Manual Mode 22:07:31 Session Resumed Session Resumed (Live, 2022-10-13 S01046, ImageSaveNum: 1046011) 22:07:59   Obs.Manager Started Obs.Manager started 22:08:01   Obs.Overseer Started Obs.Overseer started 22:08:04   Services Started Observatory Services started 22:09:19   Services Started Observatory Services started 22:09:43   Telescope Connected Telescope connected (TheSky6) 22:10:44   Camera1 Connected SBIG Camera connected (set point -10°C) 22:11:38 Session Monitoring AutoStart monitoring for Live Session opportunity between 22:11 & 05:39 22:11:40 Observatory (AutoStart) Observatory placed in Auto-Start Mode 22:11:42   AutoStart Delayed Session delayed due to cloud 23:22:43 Session AutoStarting Session autostarting (23:22) 23:23:04   Services Started Observatory Services started 23:23:09 Observatory (Auto) Observatory placed in Fully-Automated Mode 23:23:13 Session Pending Session pending (2022-10-13) 23:23:15 Session Initiating Session initiating (2022-10-13) 23:23:19   Plan Loaded Observing Plan loaded to queue (Plan ID: 808) 23:23:34   Camera1 Connected SBIG Camera connected (set point -10°C) 23:23:40   Telescope Connected Telescope connected (TheSky6) 23:23:59 Session Equilibration Session ready to Open Dome 23:24:01   Wait On Weather Waiting for acceptable conditions 23:24:03 Session Suspended Session suspended 23:24:05   Dome Closed Dome already closed 23:24:07   Queue Started Observing Queue started (25 targets selected) 23:26:13 Session Resuming Session resuming 23:26:59   Dome Opened Dome opened (opening time 45s) 23:27:01 Session Running Session running 23:27:03   Queue Resumed Observing Queue resumed 23:27:05     Target Started (8/32) Target started (8/32, AT2022oar) 23:27:15   Dome Unparked Dome unparked 23:34:12     Target Completed Target partially completed (8/32, AT2022oar) 23:34:16     Target Started (9/32) Target started (9/32, AT2022xij (CV)) 23:41:20   SoftSuspend Called Soft Suspend is called due to Deteriorating Conditions (too few stars) 23:43:56   SoftSuspend Cancelled Soft Suspend is cancelled due to improved conditions 23:45:52     Target Completed Target completed (9/32, AT2022xij (CV)) 23:45:56     Target Started (10/32) Target started (10/32, GCVS S Cyg) 23:48:39       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S) 23:52:49       Focusing Failed Foc1 focusing failed (failed QC check - Ambiguous) 23:53:07   SoftSuspend Called Soft Suspend is called due to Deteriorating Conditions (too few stars) 00:00:44     Target Completed Target completed (10/32, GCVS S Cyg) 00:00:46   Queue Paused Queue paused for Session Suspension 00:00:48 Session Suspended Session suspended 00:01:33   Dome Closed Dome closed (closing time 45s) 00:14:19 Session Resuming Session resuming 00:15:04   Dome Opened Dome opened (opening time 45s) 00:15:06 Session Running Session running 00:15:08   Queue Resumed Observing Queue resumed 00:15:11     Target Started (11/32) Target started (11/32, NGC 5894 w/SN2022pgf) 00:24:00     Target Completed Target partially completed (11/32, NGC 5894 w/SN2022pgf) 00:24:12     Target Started (12/32) Target started (12/32, GCVS CY Lyr) 00:33:21     Target Completed Target completed (12/32, GCVS CY Lyr) 00:33:33     Target Started (13/32) Target started (13/32, NGC 6745 w/SN2022prr) 00:43:42     Target Completed Target completed (13/32, NGC 6745 w/SN2022prr) 00:53:37     Target Started (14/32) Target started (14/32, GCVS SS Cyg) 01:00:12       Focusing Started-Foc1 Foc1 Focusing Started (TCF-S) 01:07:29       Focusing Failed Foc1 focusing failed (failed QC check - Quadratic) 01:15:25     Target Completed Target completed (14/32, GCVS SS Cyg) 01:15:37     Target Started (15/32) Target started (15/32, GCVS BL Lac) 01:20:32   SoftSuspend Called Soft Suspend is called due to Deteriorating Conditions (too few stars) 01:24:25     Target Completed Target partially completed (15/32, GCVS BL Lac) 01:24:27   Queue Paused Queue paused for Session Suspension 01:24:30 Session Suspended Session suspended 01:25:18   Dome Closed Dome closed (closing time 50s) 01:30:55     Target Cancelled Target cancelled (16/32, AT2022fnb (CV)) due to cloud 01:39:45 Session Resuming Session resuming 01:40:30   Dome Opened Dome opened (opening time 45s) 01:40:32 Session Running Session running 01:40:34   Queue Resumed Observing Queue resumed 01:40:36     Target Started (17/32) Target started (17/32, AT2022whd (CV)) 01:48:22     Target Completed Target partially completed (17/32, AT2022whd (CV)) 01:48:34     Target Started (18/32) Target started (18/32, GCVS Z And) 02:00:54     Target Completed Target partially completed (18/32, GCVS Z And) 02:01:06     Target Started (19/32) Target started (19/32, UGC 3923 w/SN2022rqc) 02:02:18   SoftSuspend Called Soft Suspend is called due to Deteriorating Conditions (too few stars) 02:05:06     Target Failed Target failed due to cloud (19/32, UGC 3923 w/SN2022rqc) 02:05:08   Queue Paused Queue paused for Session Suspension 02:05:10 Session Suspended Session suspended 02:05:53   Dome Closed Dome closed (closing time 45s) 02:12:48     Target Cancelled Target cancelled (19/32, UGC 3923 w/SN2022rqc) due to cloud 02:28:48     Target Cancelled Target cancelled (20/32, Nova Per 2020) due to cloud 02:49:18     Target Cancelled Target cancelled (21/32, AT2022vxe) due to cloud 03:11:18     Target Cancelled Target cancelled (22/32, AT2022al) due to cloud 03:36:08     Target Cancelled Target cancelled (23/32, AT2022kl (CV)) due to cloud 03:56:08     Target Cancelled Target cancelled (24/32, AT2022tru (CV)) due to cloud 04:07:36     Target Cancelled Target cancelled (25/32, GCVS EV Lac) due to cloud 04:26:46     Target Cancelled Target cancelled (26/32, NGC 7448 w/SN2022wsp) due to cloud 04:37:44     Target Cancelled Target cancelled (27/32, GCVS HH And) due to cloud 04:59:09     Target Cancelled Target cancelled (28/32, C/2020 V2 (ZTF)) due to cloud 05:19:09     Target Cancelled Target cancelled (29/32, UGC 6983 w/SN2022pru) due to cloud 05:33:09     Target Cancelled Target cancelled (30/32, 29P/Schwassmann-Wachmann) due to cloud 05:53:09     Target Cancelled Target cancelled (31/32, AT2022tmq) due to cloud 06:15:15     Target Cancelled Target cancelled (32/32, GCVS ER UMa) due to cloud 06:15:20   Queue Completed Job Queue completed 06:15:28 Session Closing Session closing 06:15:33   Dome Closed Dome already closed (closed at 01:05, closing time 45s) 06:15:56   Dome Parked Dome parked (parking time 15s), Az: 90.0 deg 06:17:19   Telescope Parked Telescope parked (parking time 80s) 06:17:34   Telescope State Scope parked Turn scope off. (Handbox) 06:17:56   Telescope Switched Off Telescope Power has been switched off via UPB Switch. 06:22:00 Obs.Monitor Frozen Obs.Monitor appears to be frozen at around 06:22 in section Update Foc1 Fields 07:43:52 Observatory (Manual) Observatory placed in Manual Mode 07:47:40 User Intervention User request to close and finish session 07:47:42 Session Closing Session closing 07:47:44   Dome Closed Dome already closed (closed at 01:05, closing time 45s) 07:47:46   Dome Parked Dome already parked (parked at 06:15, parking time 15s) 07:47:52   Services Stopped Night Services stopped 07:47:54 Session Housekeeping Session Housekeeping started (Create Fits Summary, Transfer Files) 07:48:00 Session Finished Session Finished

Session Alerts

Time     Alert Detail 21:34:30 Obs.Monitor Obs.Monitor appears to be frozen in section Update Foc1 Fields (last cycle at 21:33:58) 21:45:11 Obs.Manager Obs.Manager is frozen (last cycle at 21:43:07) 21:48:46 Job Queue Job Queue appears to be frozen (last IamAlive message at 21:38:12) 06:22:34 Obs.Monitor Obs.Monitor appears to be frozen in section Update Foc1 Fields (last cycle at 06:22:00) 07:47:00 Obs.Monitor Obs.Monitor has resumed again after 84.9 mins

Back to Top

Operational Issues (2022-10-13, S1046)

• Critical Issues (0)
• Major Issues (2)
• Minor Issues (3)
• Small Defects (0)
• Continuous Improvement (2)
  

[ Prev | Next ]

Critical Issues

• None

Major Issues

• ObsMonitor, ObsManager & Job Queue all became frozen during Target 2/32 (21:34- 21:43).  After slewing to Target 2/32 (WDS GRB 34), the CCD Temperature Set Point was changed from -10°C to -15°C  and before centering the target the job queue entered a wait loop at 21:33:07 in order to wait for the CCD to stabilise at  -15°C. At  21:34:05 the ObsMonitor loop was logged as being overdue, and then subsequently logged/reported as being frozen in section 'Update Foc1 Fields', with a last cycle time at at 21:33:58.  The wait loop seemingly timed out at 21:38:08 and the Job Queue resumed with preparations to take first Locate frame (but there are no indications that this first Locate frame was actuall started). 3 minutes passed by and conditions deteriorated leading to a soft suspend being called at 21:41:47.  After another minute of so conditions then became critical and a hard suspend was called by ObsManager at 21:43:07 and the session was suspended. After indicating that the Dome would be closed, the ObsManager froze with a last cycle time of 21:43:07).    At 21:46,  the ObsOverseer called the routine to 'Kill & Restart CCDSoft',  but the ObsMonitor and ObsManager remained frozen. With critical conditions and the ObsManager recognised as being frozen, the ObsOverseer stepped in and called the MakeObservatorySafe() routine, which proceeded to close the observatory dome at 21:48.  With the ObsManager frozen a User Intervention was attempted to try to regain control and restart operations. Unable to stop the Job Queue an attempt was made to Kill the task, but this merely led to the AstroMain GUI locking up, and necessiated killing AstroMain program and then stopping and restarted all observatory software.    It is suspected that there was a problem with the SBIG Camera/Driver,  but this can not be proven, and it is unclear how this can be fxied to prevent this issue in future.  The key thing is that the ObsOverseer successfully stepped and closed the shutter to make the observatory safe. Closed after review 2022-10-14.
  Closed after review  2022-10-14 (AstroMain, SBIG Camera).
   
• SBIG Camera appears to have froze/stopped responding during Target 10/32 (23:49).  After restarting the observatory software and session, an autostart was requested by user at 22:11, but couldn't be immediated acted upon due to 'too few stars', At 23:22 conditions were eventually suitable and autostart operations began at 23:23. Beginning with Target 8/32 (AT2022oar) the target was centered ok with no problems, but there was only time to taking a one out of five 180s image frames. Target 9/32 (AT2022xij) was then successfully completed (apart from guiding issues due to cloud).  Target 10/32 (GCVS S Cyg) was then begun and was successfully centered. Whilst focusing the star there seems to have been some issue with the SBIG Focus frames for Step 10/14 onwards.   The FWHM & Max Values reported are unchanged from those of Step 9/14, suggested that the Focus Profile routine isn't working with a fresh image, but is working with an older image.  The Focus Profile chart shows the last points are flat lined. Logs show that TakeImageAsync routine is seemingly frozen whilst taking step 10/14 and doesn't release the flag indicating that Camera1 is not currently accessible.  Logs show that a 'Command in progress,  Error code = 219 (0xdb)'  exception from Camera.TakeImage() during step 11/14. and all following attempts to take an image. 
  This issue requires SBIG Camera to be rebooted.   Whilst this can be performed by user but only if he is aware of the problem (but in this case the session was being conducted under fully automated control without user monitoring).    AstroMain should take various actions when this problems occurs:  a) AstroMain needs to be modified to raise an Red alert when the 'Command in progress,  Error code = 219'  is first seen. 2) A kill and Restart CCDSoft could be tried, however from past experience this most likely doesn't fix the problem. 3) AstroMain initiates an action to perform an automated reboot of the SBIG Camera.  At present 3) isn't possible, due to issues trying to get code work that could allow Combee II adapter to communicate with and operate the HUE Smart Plug that the SBIG Camera takes it Power from.   4) Give up running new targets whilst the Error Code = 219 problem persists.   It is possible that a 'precautionary' User reboot of the SBIG Camera made before the session or after the session was restarted following problems at 21:34:-21:43 might have prevented the problem. Closed after implementing 1) in code (AstroMain 3.55.4, 2022-10-14)
  Closed  2022-10-14 (AstroMain 3.55.4,  SBIG Camera).

Minor Issues

• No ObsCam images were recorded during the session. Due to a user error the AstroObsCam program although running hadn't had the Camera Service started and the camera was connected.  Care has to be taken to ensure the AstroObsCam program is both started and the Camera Service started.
  Issue understood 2022-10-14 (AstroObsCam, AllSky/Weather Computer).
   
• Focus Profile showed data points for last 5 focus steps (10/14 to 14/14) even though a Focus Frame image wasn't acquired. Due to a glitch with SBIG camera during Focus Profile for Target 10/32 (GCVS S Cyg) focus frame images weren't acquired for the final five focus steps. Despite this FWHM, HFD and MaxValue values were recorded with values taken from last valid focus frame (Step 9/14) and used in an attempt to get an inappropriate Best Focus position. Fortuanetly due to an ambiguous result the QC Checks failed and the Focus Offset wasn't modified.  When the focus frame failed FWHM, HFD and Max Value should have been set to vcNull and the marked as non-Valid.
  Fixed  2022-10-20 (AstroMain 3.55.8).
   
• Job Queue continued to execute targets even though each attempt to take any image with SBIG Camera was failing with exception. Whilst taking Focus Frame image for step 10/14 for Target 10/32 (GCVS S Cyg), and all subseuqent Take Image attempts (focus frames, locate frames and main frames) encountered exceptions messages  'Command in progress,  Error code = 219 (0xdb)'. Job Queue executor continued processing targets and marked them as completed and produced 'invalid' JPG pictures based off last image (Focus Frame from step 9/14) even though no images were acquired.  TakeImageASync() routine has been modified in AstroMain 3.55.4 to set a flag when this exception occurs and increment a counter.   Job Executor should now be modified to use these flags and stop the execution of new targets until the flag becomes reset.  Partial fix made in AstroMain 3.55.4 (2022-10-14) which ignores the call to create JPG pictures if bSbigDriverException is raised and sets FailureState.SbigCamera (29).  Issue considered to be closed for now given improvements made and improved stability in sessions S1057/S1058
  Partially resolved 2022-10-14 (AstroMain 3.55.6).

Small Defects

• None

• Add additional FailureState enum value to AstroMain and AstroPlan to record observing failures due to SBIG Cameras. Add FailureState.SbigCamera = 29. to FailureState enum list.
  Done 2022-10-14 (AstroMain 3.55.4, AstroPlan 1.32.6).
   
• Add automated operation of Conbee 2 ( USB gateway) to turn off/turn on HUE Philips Smart Plugs in order to reboot devices like the SBIG Camera. Conbee 2 is a universal Zigbee USB gateway which offers the potential to be able to control the two Hue Philips Smart Plugs used in the observatory in order to reboot the SBIG Camera or Pulsar Dome in order to resolve issues that might occassional prevent the successful operation of the observatory.  A Conbee 2 was purchased last year but initial attempt to achieve programmed access to the Conbee 2 and to Hue Smart Plugs wasn't successful and the operation was put on the back burner.  With the problem in session S1046 which would have benefitted from automated reboot of the SBIG Camera it is time to resurrect the project again and get it to work.  Conbee 2 website is https://phoscon.de/en/conbee2 . The Conbee 2 would most-likely be located eventually on the USB Hub attached to the Observatory Computer in Bay 2. In the short term it would be used connected to the Development Computer and one of the Smart Plugs brought indoors for testing.  AstroMain 3.55.7 contains a prototype implementation using Deconz (which has to be running) and a Rest API to access PhosCon App. Delivered around 2022-10-21.
  Done 2022-10-21 (AstroMain,  AstroConbee).

[ Prev | Next ]

Back to Top