Panacea Journal of Medical Sciences
Panacea Journal of Medical Sciences (PJMS) open access, peer-reviewed triannually journal publishing since 2011 and is published under auspices of the “NKP Salve Institute of Medical Sciences and Research Centre”. With the aim of faster and better dissemination of knowledge, we will be publishing the article ‘Ahead of Print’ immediately on acceptance. In addition, the journal would allow free access (Open Access) to its contents, which is likely to attract more readers and citations to articles published in PJMS.Manuscripts must be prepared in accordance with “Uniform requiremen...

Thromboelastometric evaluation of changes in coagulation dynamics following therapeutic plasma exchange using different replacement fluids
Abstract
Background: Therapeutic plasma exchange (TPE) decreases clotting factors and cellular blood components. Choice of replacement fluid can be important factor in outcome of therapeutic large volume plasma exchange.ObjectiveThis study aimed to evaluate changes in coagulation parameers using thromboelastography (TEG) prior and post-plasma exchange with albumin alone or with fresh frozen plasma (FFP). DesignThis is prospective comparative study.
Materials and Methods: A prospective comparative study carried out in the Department of Transfusion Medicine, Indraprastha Apollo Hospitals, New Delhi, from 1st September 2016 to 31st December 2017 in the patients of renal allograft recipient or post renal transplant.
Results: A total of 47 chronic kidney disease(CKD) patients were included.TEG analysis before and after TPE procedures revealed a relatively hypo-coagulable state when albumin only was used as a replacement fluid. The R (3.489±0.9136 vs 7.669±1.695) and K (3.09±0.50 vs 6.611±1.588) increased while alpha angle value (38.10±7.83 vs 34.92±7.564) decreased (p value & lt; 0.001).The mean increase in R value was more pronounced (4.18 vs 0.39) in TPE with albumin only compare to TPE with albumin and FFP group (p value & lt; 0.005).
Conclusion: Blood coagulation parameters measured by TEG analysis showed a significant decrease in clotting factors in patients where TPE was performed with only albumin as replacement fluid.
Introduction
Therapeutic plasma exchange (TPE) is a procedure in which patient’s blood is passed through a automated cell separator where blood is being centrifuged and separates plasma from other components of blood.[1] The plasma is then removed, disposed of, and replaced with either a colloid solution or a combination of crystalloid and colloid solutions.[1] The selection of replacement fluid can have a significant impact on the therapeutic plasma exchange (TPE) outcome for high volumes. TPE can result in bleeding diathesis in recipients of renal transplants by altering the platelets and coagulation system, which are already significantly altered.[2]
If only albumin is used as replacement fluid after TPE, there is a significant decrease in fibrinogen, other procoagulants, and anticoagulant proteins as evidenced by the prolongation of the activated partial thromboplastin time (aPTT) and prothrombin time (PT), even though there are no severe hemorrhagic episodes following TPE procedures.[3] After TPE, the majority of coagulant proteins recover to 85–100% normal in a day, whereas factor X and fibrinogen take two days to recover.[4], [5], [6]
A point-of-care tool thromboelastography (TEG) uses changes in viscoelastic properties during the clotting process. With a graphic depiction of the fibrin polymerization, TEG provides information regarding the dynamics of clot formation, strength, and stability.[7], [8], [9] Because TEG can provide data at the patient's bedside in less than ten minutes, it has been validated as a tool for managing perioperative bleeding. TEG may lessen the number of needless haemostatic blood product transfusions and help medical professionals differentiate between coagulopathy and surgical bleeding causes. The test might also be used to prevent hesitancy in seeking help and lower the number of surgical reoperations brought on by bleeding. In the end, mortality could be decreased.[10], [11], [12]
The management of bleeding during surgery and monitoring of coagulation activity of renal transplant recipients can be difficult. The standard tests like activated partial thromboplastin time(APTT) PT(partial thromboplatin time),International Normalized Ratio (INR) may not be as accurate as point-of-care (POC) assays like TEG, which measure the viscoelastic properties of blood.[13] In order to more accurately estimate the patient's bleeding risk, coagulation is assessed in whole blood and includes interactions with red blood cells and platelets.[13] This provides extra information on the patient's coagulation status.
Aim and Objective
Our study's objective is to examine the alterations in the coagulation profile as determined by TEG before and after plasma exchange using albumin as the replacement fluid with/without fresh frozen plasma.
Materials and Methods
A prospective comparative study was carried out in the Department of Transfusion Medicine, Indraprastha Apollo Hospitals, New Delhi, from 1st September 2016 to 31st December 2017 in the patients of renal allograft recipient or post renal transplant with history of CKD. The study was done after approval from institution’s ethical committee.
Inclusion criteria
In this study Pre-renal transplant and post-renal transplant patients with history of CKD requiring plasma exchange will be included.
Exclusion criteria
In this study Patients with pre-existent coagulopathies, Patients on anticoagulant and antiplatelet therapies, Combined liver and kidney transplant patients and Patients requiring plasma exchange with use of FFP only as a replacement fluid will be excluded from the study.
Patient selection process
The two groups of patients Group A and Group B. were formed after taking an informed consent.The patients who underwent TPE with 5% albumin only as a replacement fluid were placed in group A, while patients who underwent TPE with 5% albumin and FFP as a replacement fluid were placed in group B. The decision for replacement fluid was made by clinician discretion.
Therapeutic plasma exchange
TPE was performed using Haemonetics MCS plus (Braintree, USA) cell separator (intermittent centrifugation) using central venous access (jugular venous catheter or femoral venous catheter) or arterio-venous fistula. A single use, sterile disposable set, REF 981E (Haemonetics, USA) was used. ACD-A was used as anticoagulant with ACD-A: blood ratio of 1:14-1:16. A total of 1.5 calculated plasma volume was exchanged per session.
Sample for thromboelastography
Patient’s blood sample for TEG testing was taken before starting the procedure and subsequently 45 min after the end of each TPE procedure. Initially, the plain vial was used to take 1- 2 ml of patient’s blood sample and then it was immediately transferred to kaolin vial with which it was mixed properly and then with the use of a pipette, 340 μL kaolin activated whole blood sample was run in TEG (TEG Hemostasis System 5000 series (TEG®) as provided by Haemoscope Corporation, Niles, Illinois, USA). The analyzer was positioned on a stable bench distant from sources of vibration.machine for evaluation.
Statistical analysis
Continuous variables were summarized as mean ± SD whereas nominal/categorical variables were presented as proportions (%). Chi-Square test and Fisher exact test were used for analysis of nominal/categorical variables while continuous variables were analysed by using Unpaired t-test & Paired t-test. Mann-Whitney Rank Sum Test was used for ordinal variables or not normally distributed continuous variables. ‘p’ value < 0.05 was taken as significant. Medcalc 16.4 version software was used for all statistical calculations.
Results
Total 47 patients with a history of CKD (pre-transplant and post-transplant) were included in the study who underwent a total of 71 TPE procedures as part of desensitization protocol. In 35 procedures (Group A) 5% albumin was used as a replacement fluid while in other 36 procedures (Group B) 5% albumin with FFP was used as a replacement fluid. There was no bleeding episodes during study period in group A and group B.
Patient demographic details
A total of 23 patients underwent 35 TPE procedures with albumin only as a replacement fluid while 24 patients underwent 36 TPE procedures with albumin and FFP as a replacement fluid. The mean age was 44±9.4 years in TPE with albumin only group while mean age was 42±10.9 years in TPE with albumin and FFP group. As shown age had no statistical significance in result outcome of these two groups (P = 0.760). A number of males were 15 and females were 8 in TPE with albumin only group while a number of males was 17 and females were 7 in TPE with albumin and FFP group ([Table 1]).
|
TPE with albumin only |
TPE with albumin and FFP |
Number of patients |
23 |
24 |
Number of TPE |
35 |
36 |
*Age [mean± SD] |
44±9.4 |
42±10.9 |
Male |
15 |
17 |
Female |
8 |
7 |
Therapeutic plasma exchange details
hows the TPE details including plasma volume (PV) extracted, total volume extracted in a litre(L), anticoagulant ACD and 0.9% normal saline use in two groups. As shown none of the parameters had the significance in the resulting outcome of these two groups.
|
TPE with albumin only |
TPE with albumin and FFP |
‘p’ value |
PV extracted |
1.5 PV |
1.5 PV |
- |
Volume extraction [mean± SD] |
4.10L±0.5 L |
3.96L±0.4 L |
0.198** |
Replacement % mean |
92.22±6.375% |
89.41±9.192% |
0.206* |
ACD used [mean± SD] |
466 ml±83 ml |
435 ml±80 ml |
0.114 |
Normal Saline used [mean± SD] |
675 ml±275 ml |
780 ml±254 ml |
0.099 |
Albumin used [mean] |
606 ml |
590 ml |
- |
Comparison of TEG parameters before and after TPE using albumin only
As shown in table 3, there was mean increase in R time, K time, MA value and LY30% value while a mean decrease in alpha angle value after TPE which was statistically significant finding in TEG coagulation parameters changes after TPE in this group (p value< 0.001). The mean increase in R time and K time while a mean decrease in alpha angle value after TPE was suggestive of hypocoagulable state. The mean increase in MA value after TPE did not correlate with this hypocoagulable TEG tracing.
|
Before TPE |
After TPE |
‘p’ value* |
R time [mean± SD] |
3.489±0.9136 |
7.669±1.695 |
<0.001 |
K time [mean± SD] |
3.125±0.6267 |
6.611±1.588 |
<0.001 |
Alpha angle degree [mean± SD] |
37.17±8.799 |
34.92±7.564 |
<0.001 |
MA mm [mean± SD] |
36.55±12.09 |
47.99±11.14 |
<0.001 |
LY30 % [mean± SD] |
0.2361±0.5144 |
1.1±1.273 |
<0.001 |
Comparison of TEG parameters before and after TPE using albumin and FFP
As shown in [Table 4], there was mean significant increase in R time, K time, alpha angle, MA value (p value< 0.005) after TPE and except for LY30% value (p value =0.080). The mean increase in R and K time was only suggestive of hypocoagulable state and a mean increase in alpha angle and MA value did not correlate with this finding.
|
Before TPE |
After TPE |
‘p’ value* |
R time [mean± SD] |
4.01±1.444 |
4.40±1.136 |
0.001 |
K time [mean± SD] |
3.19±0.394 |
3.58±0.551 |
<0.001 |
Alpha angle degree [mean± SD] |
37.57±6.452 |
39.16±7.341 |
<0.001 |
MA mm [mean± SD] |
35.94±9.765 |
49.96±7.843 |
<0.001 |
LY30 % [mean± SD] |
0.1486±0.4623 |
1.049±3.364 |
0.080 |
Comparison of change in TEG values before and after TPE in both the groups
The prolongation in R time, K time and MA value were more profound in TPE with albumin only as a replacement fluid (p value< 0.005). There was a decrease in alpha angle value in TPE with albumin only as a replacement fluid while its value increased in TPE with albumin and FFP as a replacement fluid (p value< 0.005).([Table 5])
|
TPE with albumin |
TPE with albumin and FFP |
P value |
Mean R time change |
+4.18 |
+0.39 |
<0.005 |
Mean K time change |
+3.486 |
+0.39 |
<0.005 |
Mean alpha angle degree change |
-2.25 |
+1.59 |
<0.005 |
Mean MA mm change |
+11.44 |
+14.02 |
<0.005 |
TEG parameters values in pre-transplant vs post-transplant patients
Pre-transplant (1st or 2nd time) renal allograft recipient group (n=47) and renal post-transplant patients (n=24). R value and LY30% value was higher in pre-transplant patients compare to post-transplant patients which were statistically significant finding (p value<0.001). The R value indicated that pre-transplant patients had hypercoagulable profile compare to post-transplant patients (3.28±0.61 vs 4.65±1.59). Other parameters did not reach the statistical significance in these two groups.
|
Pre-transplant patients |
Post-transplant patients |
‘p’ value* |
No. of patients |
47 |
24 |
- |
R time [mean± SD] |
3.28±0.61 |
4.65±1.59 |
<0.001 |
K time [mean± SD] |
3.09±0.50 |
3.30±0.54 |
0.108 |
Alpha angle degree [mean± SD] |
38.10±7.83 |
35.94±7.32 |
0.265 |
MA mm [mean± SD] |
37.92±10.38 |
32.98±11.47 |
0.071 |
LY30 % [mean± SD] |
0.05±0.24 |
0.48±0.69 |
<0.001 |
Discussion
In our study, no parameters related to patient demographics or TPE had the significance in the resulting outcome of two different replacement fluid strategy, however, age and gender variability is described by Sun et al. [14] In our study age difference did not reach a statistical significance, so it had no impact on result outcome.
In our study, compared to before TPE, patients TEG analysis after TPE procedure revealed a relatively hypocoagulable state when albumin only was used as a replacement fluid. The value of R (3.489±0.9136 vs 7.669±1.695) and K (3.09±0.50 vs 6.611±1.588) increased while alpha angle value (37.17±8.79 vs 34.92±7.564) decreased (p value< 0.001).The value of MA (36.55±12.09 vs 47.99±11.14) was suggestive of a relatively hypercoagulable state as its value increased after TPE procedure and TEG tracing did not reveal a true picture of the hypocoagulable state.Thölking et al,conducted a study on 10 patients using 5% albumin as replacement fluid and results were similar to our study. [15]
In another study by Blasi et al., 6 patients were treated with TPE with 5% albumin as replacement fluid.Clotting time (CT), maximum clot firmness (MCF), delay in clot formation, and reduction in clot firmness were all TEM parameters that fell below the usual ranges. [16] According to reports, the fibrinogen value decreased by 67% on average. This is due to the fact that fibrinogen, IgM, and a2-macroglobulin are primarily found in the intravascular compartment, and PE is highly effective in lowering these levels. This result was consistent with our investigation because the reaction rate (R) for TEG and the CT for ROTEM are both measured in minutes, or the amount of time it takes for the trace to attain an amplitude of 2 mm. and R value changed from hypercoagulable to normal state as per reference value and R time was prolonged after TPE in our study.
In our study, we suspected that increase in MA value after TPE procedure might be due to activation of PLT after coming into contact with the TPE equipment as MA measures the maximum strength (in millimeters) of the formed clot and primarily evaluates fibrinogen and platelets but further studies required to validate this point.Parikh et al. and Schoorl et al. also described the reduction in platelet count after TPE. [17], [18]
In our study, when FFP along with albumin was used as a replacement fluid, K value increased after TPE procedure but compare to table 4, K value increase was more pronounced (3.486 vs 0.39) when albumin alone was used as a replacement fluid (p value< 0.005) as high value of K commonly suggests the necessity for administration of cryoprecipitate or fibrinogen concentrates. However, no other studies have compared coagulation profile of TEG analysis in albumin and FFP as a replacement fluid with albumin alone as a replacement fluid in our best knowledge.
In our study while comparing the coagulation parameters it was obseved that there was a mean increase in R time after TPE with albumin and FFP as a replacement fluid (p value< 0.005). The mean increase in R was suggestive of hypocoagulable state and mean increase in alpha angle and MA value did not correlate with this finding. The mean increase in R value was more pronounced (4.18 vs 0.39) in TPE alone with albumin compare to TPE with albumin and FFP group (p value< 0.005).
It was suggested by Agarwal et al., and Tek et al. to monitor fibrinogen levels and to switch to plasma when they drop below 1.25 g/L. [19], [20] According to Feuring et al., FFP should be taken into consideration following PE if there is a history of hemorrhagic episodes or if fibrinogen fails to return to levels that are safe. [21] According to Lance et al. most patients have normal liver function, but even those with mild liver damage should use caution if they have cirrhosis. [22] Based on our observations, albumin usage is not recommended in liver patients at such high risk. According to Pilgeram et al. the condition of severe hypofibrinogenemia may last longer than it would given a fibrinogen production rate of 0.26–0.38 g/l per day. [23]
In our study group, there was a mean decrease in alpha angle after TPE with albumin only while mean increase after TPE noted in TPE with albumin and FFP (-2.25 vs +1.59) (p value <0.005). This might be due to fibrinogen factor in FFP which increased the alpha angle value after TPE procedure. As well as a mean increase in MA value was more in TPE with albumin and FFP compared to TPE with albumin only (+14.02 vs +11.44) (p value <0.005) and again fibrinogen factor might be responsible for this TEG tracing. Pre-transplant patients were hypercoagulable while post-transplant patients were hypocoagulable as TEG tracing showed in our study (3.28±0.61 vs 4.65±1.59) (p value< 0.001). Other parameters did not reach statistical significance except LY30%.
Sabovic et al., and Vaziri et al. described the coagulopathy in pre-transplant patients on multiple hemodialysis (MHD). [24], [25] As per Sabovic et al., compared to CKD patients who are not receiving dialysis, hemodialysis patients exhibit a moderation of the hypercoagulable profile, with a lower MA and an increased clot breakdown. In our study also pre transplant patients on MHD had decreased MA value (37.92±10.38) compare to reference value. [16], [17], [24]
Finding in our study was similar to these findings reported by Timmermans et al. as R value was lower in pre transplant patients (3.28±0.61) compare to post-transplant patients (4.65±1.59) (p<0.001).[26] We could not establish the significance of LY30% value in TEG analysis before and after TPE procedures as 42 patients had no fibrinolysis in TEG analysis and there is no literature available on LY30% value significance in TPE procedures in best of our knowledge. Further studies are required to validate its significance in TPE procedurres.
Limitations
This study has small sample size single center study including CKD patients. We di not do standard coagulation tests to compare it with thromboelastography parameters. Significant intra- and inter-laboratory variability can arises from the lack of standardization in the approach for Kaolin Activated Whole Blood Thromboelastography.
Conclusion
Our observations led us to the conclusion that, in thromboelastography analysis, TPE with albumin alone without the inclusion of fresh frozen plasma can seriously disrupt blood coagulation. If albumin is the only replacement fluid used after a kidney transplant, the risk of bleeding increases. For CKD patients who are at a high risk of bleeding, thromboelastography seems to be a practical coagulation profile test.
Confict of Interest
None.
Source of Funding
None.
References
- Schwartz J, Winters J, Padmanabhan A, RB, Delaney M, Linenberger M. Guidelines on the Use of Therapeutic Apheresis in Clinical Practice—Evidence-Based Approach from the Writing Committee of the American Society for Apheresis: The Sixth Special Issue. J Clin Apher. 2013;28(3):145-284. [Google Scholar]
- Jalal D, Chonchol M, Targher G. Disorders of Hemostasis Associated with Chronic Kidney Disease. Semin Thromb Hemost. 2010;36(1):34-40. [Google Scholar]
- Cid J, Carbassé G, Andreu B, Baltanás A, Garcia-Carulla A, Lozano M. Efficacy and safety of plasma exchange: an 11-year single-center experience of 2730 procedures in 317 patients. Tranfus Apher Sci. 2014;51(2):209-14. [Google Scholar]
- Chirnside A, Urbaniak S, Prowse C, Keller A. Coagulation abnormalities following intensive plasma exchange on the cell separator. II. Effects on factors I, II, V, VII, VIII, IX, X and antithrombin III. Br J Haematol. 1981;48(4):627-34. [Google Scholar]
- Sultan Y, Bussel A, Maisonneuve P, Poupeney M, Sitty X, Gajdos P. Potential danger of thrombosis after plasma exchange in the treatment of patients with immune disease. Transfusion. 1979;19(5):588-93. [Google Scholar]
- Keller A, Chirnside A, Urbaniak S. Coagulation abnormalities produced by plasma exchange on the cell separator with special reference to fibrinogen and platelet levels. Br J Haematol. 1979;42(4):593-603. [Google Scholar]
- Jackson G, Ashpole K, Yentis S. The TEG vs the ROTEM thromboelastography/ thromboelastometry systems. Anaesthesia. 2009;64(2):212-5. [Google Scholar]
- Luddington R. . Clin Lab Haematol. 2005;27(2):81-90. [Google Scholar]
- Bowbrick V, Mikhailidis D, Stansby G. The use of citrated whole blood in thromboelastography. Anesth Analg. 2000;90(5):1086-8. [Google Scholar]
- Wikkelsoe A, Afshari A, Wetterslev J. Monitoring patients at risk of massive transfusion with Thrombelastography or Thromboelastometry: a systematic review. Acta Anaesthesiol Scand. 2011;55(10):1174-89. [Google Scholar]
- Hagemo J, Naees P, Johansson P, Windeløv N, Cohen M, Røislien J. Evaluation of TEG(®) and RoTEM(®) inter-changeability in trauma patients. Injury. 2012;44(5):600-5. [Google Scholar]
- Dirkmann D, Gorlinger K, Peters J. Assessment of early thromboelastometric variables from extrinsically activated assays with and without aprotinin for rapid detection of fibrinolysis. Anesth Analg. 2014;119(3):533-42. [Google Scholar]
- Ganter M, Hofer C. Coagulation monitoring: current techniques and clinical use of viscoelastic point-of-care coagulation devices. Anesth Analg. 2008;106(5):1366-75. [Google Scholar]
- Sun J, Bian M, Zhong T, Lu Y, Zhu B, Wen H. Reference values for kaolin activated thromboelastography in volunteers of Anhui Province in China. J Clin Lab Anal. 2017;31(6). [Google Scholar] [Crossref]
- Thölking G, Mesters R, Dittrich R, Pavenstädt H, Kümpers P, Reuter S. Assessment of Hemostasis after Plasma Exchange Using Rotational Thrombelastometry (ROTEM). PLoS One. 2015;10(6). [Google Scholar] [Crossref]
- Blasi A, Cid J, Beltran J, Taurà P, Balust J, Lozano M. Coagulation profile after plasma exchange using albumin as a replacement solution measured by thromboelastometry. Vox Sang. 2016;110(2):159-65. [Google Scholar]
- Parikh A, Spencer F, Lessard D, Emery C, Baylin A, Linkletter C. Venous thromboembolism in patients with reduced estimated GFR: a population-based perspective. Am J Kidney Dis. 2011;58(5):746-55. [Google Scholar]
- Schoorl M, Schoorl M, Nubé M, Bartels P. Coagulation activation, depletion of platelet granules and endothelial integrity in case of uraemia and haemodialysis treatment. BMC Nephro. 2013;14. [Google Scholar] [Crossref]
- Agarwal S, Coakley M, Reddy K, Riddell A, Mallett S. Quantifying the effect of antiplatelet therapy: A comparison of the platelet function analyzer (PFA-100) and modified thromboelastography (mTEG) with light transmission platelet aggregometry. Anesthesiology. 2006;105(4):676-83. [Google Scholar]
- Tek I, Arslan O, Arat M, Ayyildiz E, Tol M, Oral M. Effects of replacement fluids used for therapeutic plasma exchange on plasma viscosity and plasma oncotic pressure. Transfusion and apheresis science. Transfus Apher Sci. 2004;31(2):89-93. [Google Scholar]
- Feuring M, Gutfleisch A, Ganschow A, Richter E, Eichler H, Dempfle C. Impact of plasmapheresis on platelet hemostatic capacity in healthy voluntary blood donors detected by the platelet function analyzer PFA-100. Platelets. 2001;12(4):236-40. [Google Scholar]
- Lance M, Nigivaggi M, Schols S, Feijge M, Oehrl S, Kuiper G. Perioperative dilutional coagulopathy treated with fresh frozen plasma and fibrinogen concentrate: a prospective randomized interventional trial. Vox Sang. 2012;103(1):25-34. [Google Scholar]
- Pilgeram L. Control of fibrinogen biosynthesis: role of the FFA/albumin ratio. Cardiovasc Eng. 2010;10(2):78-83. [Google Scholar]
- Sabovic M, Salobir B, Zupan I, Bratina P, Bojec V, Ponikvar J. The influence of the haemodialysis procedure on platelets, coagulation and fibrinolysis. Pathophysiol Haemost Thromb. 2005;34(6):274-8. [Google Scholar]
- Vaziri N, Gonzales E, Wang J. Blood coagulation, fibrinolytic, and inhibitory proteins in end-stage renal disease: effect of hemodialysis. Am J Kidney Dis. 1994;23(6):828-35. [Google Scholar]
- Timmermans J, Dolmans G, Kahn D. Sequential changes in coagulation after renal transplantation: A thromboelastogram study. Dial Transplant. 2009;38(4):133-8. [Google Scholar]
- Abstract
- Introduction
- Aim and Objective
- Materials and Methods
- Inclusion criteria
- Exclusion criteria
- Patient selection process
- Therapeutic plasma exchange
- Sample for thromboelastography
- Statistical analysis
- Results
- Patient demographic details
- Therapeutic plasma exchange details
- Comparison of TEG parameters before and after TPE using albumin only
- Comparison of TEG parameters before and after TPE using albumin and FFP
- Comparison of change in TEG values before and after TPE in both the groups
- TEG parameters values in pre-transplant vs post-transplant patients
- Discussion
- Limitations
- Conclusion
- Confict of Interest
- Source of Funding
- References
Article Metrics
- Visibility 8 Views
- Downloads 3 Views
- DOI 10.18231/pjms.v.15.i.1.48-53
-
CrossMark
- Citation
- Received Date March 15, 2024
- Accepted Date April 18, 2024
- Publication Date March 12, 2025