Question : rst.MoveLast taking forever!

Hello all,

I was under the impression that recordsets were supposed to be a fast way to move around within data. The attached function is taking forever to run. It's stuck on rst.MoveLast. The table Demand_Check_C has several million records but it's taking an hour to get to the 2Mth record. Is this normal? 
1:
2:
3:
4:
5:
6:
7:
8:
9:
10:
11:
12:
13:
14:
15:
16:
17:
18:
19:
20:
21:
22:
23:
24:
25:
26:
27:
28:
29:
30:
31:
32:
33:
34:
35:
36:
37:
38:
39:
40:
41:
42:
43:
44:
45:
46:
47:
48:
49:
50:
51:
52:
53:
54:
55:
56:
57:
58:
59:
60:

'Speed up qry10002 with literals
Public Function GetDemand()
    Dim dbs As Database
    Dim qdf As DAO.QueryDef
    Dim rst As DAO.Recordset
    Dim sInsert As String, sSelect As String, sFrom As String, sWhere As String, sSQL As String
    Dim lRecordCount As Long: lRecordCount = 0
    Dim lCounter As Long
    
    DoCmd.SetWarnings False
    
    DoCmd.RunSQL "DELETE * FROM tblDemand"

    'sInsert = "INSERT INTO tblDemand"
    sSelect = " SELECT LOWES.T2398_IFM_FRC_VAL.T024_ITM_NBR, LOWES.T2398_IFM_FRC_VAL.T063_LCT_NBR, CASE WHEN SEN_BGN_WK_NBR<=SEN_END_WK_NBR THEN SEN_END_WK_NBR-SEN_BGN_WK_NBR+1 ELSE 52-SEN_BGN_WK_NBR+SEN_END_WK_NBR+1 END AS ""Selling Weeks"", " & _
    " LOWES.T2355_SEN_PRL.WK1_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK2_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK3_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK4_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK5_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK6_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK7_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK8_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK9_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK10_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK11_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK12_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK13_BAS_IND_FCT AS ""Q1 Sum of BIs""," & _
    " LOWES.T2355_SEN_PRL.WK14_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK15_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK16_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK17_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK18_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK19_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK20_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK21_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK22_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK23_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK24_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK25_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK26_BAS_IND_FCT AS ""Q2 Sum of BIs""," & _
    " LOWES.T2355_SEN_PRL.WK27_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK28_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK29_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK30_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK31_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK32_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK33_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK34_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK35_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK36_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK37_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK38_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK39_BAS_IND_FCT AS ""Q3 Sum of BIs""," & _
    " LOWES.T2355_SEN_PRL.WK40_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK41_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK42_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK43_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK44_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK45_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK46_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK47_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK48_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK49_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK50_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK51_BAS_IND_FCT+LOWES.T2355_SEN_PRL.WK52_BAS_IND_FCT AS ""Q4 Sum of BIs"""
    sFrom = " FROM (LOWES.T2354_ITM_LCT_PRL INNER JOIN LOWES.T2355_SEN_PRL ON LOWES.T2354_ITM_LCT_PRL.T2355_PRL_TAG_ID = LOWES.T2355_SEN_PRL.T2355_PRL_TAG_ID) INNER JOIN LOWES.T2398_IFM_FRC_VAL ON (LOWES.T2354_ITM_LCT_PRL.T063_LCT_NBR = LOWES.T2398_IFM_FRC_VAL.T063_LCT_NBR) AND (LOWES.T2354_ITM_LCT_PRL.T024_ITM_NBR = LOWES.T2398_IFM_FRC_VAL.T024_ITM_NBR)"
    sWhere = " WHERE (LOWES.T2398_IFM_FRC_VAL.T024_ITM_NBR = -1 AND LOWES.T2398_IFM_FRC_VAL.T063_LCT_NBR = -1)"

    Set dbs = CurrentDb()
    Set rst = dbs.OpenRecordset("SELECT T024_ITM_NBR, T063_LCT_NBR FROM Demand_Check_C;")

    If rst.EOF Then
        lRecordCount = 0
    Else
        rst.MoveLast
        lRecordCount = rst.RecordCount
    End If

    rst.MoveFirst

    Do While Not rst.EOF

        lCounter = lCounter + 1
        sWhere = sWhere & " OR (LOWES.T2398_IFM_FRC_VAL.T024_ITM_NBR = " & rst("T024_ITM_NBR") & " AND LOWES.T2398_IFM_FRC_VAL.T063_LCT_NBR = " & rst("T063_LCT_NBR") & ")"

        If lCounter > 60 Or rst.AbsolutePosition = lRecordCount - 1 Then
            sSQL = sInsert & sSelect & sFrom & sWhere ' & " WITH UR"
            Set qdf = CurrentDb().QueryDefs("qryPassThrough")
            qdf.SQL = sSQL

            DoCmd.OpenQuery "qryDemand"

            'DoCmd.RunSQL sSQL
            lCounter = 0
            sWhere = " WHERE (LOWES.T2398_IFM_FRC_VAL.T024_ITM_NBR = -1 AND LOWES.T2398_IFM_FRC_VAL.T063_LCT_NBR = -1)" '-1 = dummy # to deal with 1st OR
        End If

        rst.MoveNext

    Loop
    
    DoCmd.SetWarnings True
    
    rst.Close

End Function

Answer : rst.MoveLast taking forever!

Yes, this seems normal.

It isn't moving to the last record, it's populating all records to the last, in a dynamic recordset (you can potentially edit any record and it should reflect any editing taking place dynamically!). It's not simple, but it's useless.

If Demand_Check_C is an Access table, you can open it in table mode and you can read the last record instantly (but a table doesn't have a record cont). If it isn't, you can use dbForwardOnly to speed things up, but, as the name implies, you cannot move back.

Basically, your function might very well run for a dozen hours, It's not clear what it does, but if you are in the process of normalising a table structure, some operations can take that sort of time. But you don't need the record count.

Use a table-type or a forward only recordset, use the 60 records chunks (still yielding in the order of 100'000 non-trivial insert queries to analyse, optimise, and run), and copy the inner section outside of the loop to get the less-than-60 records from the end.

Do not use the total record count or absolute positions.

(°v°)
Random Solutions  
 
programming4us programming4us